diff --git a/README.md b/README.md index 8dbdf83..1a47d9b 100644 --- a/README.md +++ b/README.md @@ -458,6 +458,153 @@ tar -czf config-backup.tar.gz .env docker-compose.yml services/ - `ARCHITECTURE.md`: 시스템 구조 - `PIPELINE_SCHEDULER_GUIDE.md`: Pipeline 가이드 +## 하이브리드 배포 (Hybrid Deployment with K8s) + +Site11은 Docker Compose와 Kubernetes를 함께 사용하는 하이브리드 배포를 지원합니다. + +### 배포 아키텍처 + +#### Docker Compose (인프라 및 중앙 제어) +- **인프라**: MongoDB, Redis, Kafka, Zookeeper +- **중앙 제어**: Pipeline Scheduler, Pipeline Monitor, Language Sync +- **관리 콘솔**: Console Backend/Frontend + +#### Kubernetes (무상태 워커) +- **데이터 수집**: RSS Collector, Google Search +- **처리 워커**: Translator, AI Article Generator, Image Generator +- **자동 스케일링**: HPA(Horizontal Pod Autoscaler) 적용 + +### 하이브리드 배포 시작 + +#### 1. Docker 인프라 시작 +```bash +# 하이브리드 용 docker-compose 사용 +docker-compose -f docker-compose-hybrid.yml up -d + +# 상태 확인 +docker-compose -f docker-compose-hybrid.yml ps + +# 로그 확인 +docker-compose -f docker-compose-hybrid.yml logs -f pipeline-scheduler +``` + +#### 2. K8s 워커 배포 +```bash +# K8s 매니페스트 디렉토리로 이동 +cd k8s/pipeline + +# API 키 설정 (configmap.yaml 편집) +vim configmap.yaml + +# 배포 실행 +./deploy.sh + +# 배포 상태 확인 +kubectl -n site11-pipeline get pods +kubectl -n site11-pipeline get hpa +``` + +### K8s 자동 스케일링 설정 + +#### HPA (Horizontal Pod Autoscaler) 구성 +| 서비스 | 최소 Pod | 최대 Pod | CPU 임계값 | 메모리 임계값 | +|--------|---------|---------|------------|-------------| +| RSS Collector | 1 | 5 | 70% | 80% | +| Google Search | 1 | 5 | 70% | 80% | +| Translator | 2 | 10 | 70% | 80% | +| AI Generator | 1 | 8 | 70% | 80% | +| Image Generator | 1 | 6 | 70% | 80% | + +#### 스케일링 모니터링 +```bash +# HPA 상태 실시간 모니터링 +kubectl -n site11-pipeline get hpa -w + +# Pod 수 확인 +kubectl -n site11-pipeline get pods -o wide + +# 메트릭 상세 정보 +kubectl -n site11-pipeline describe hpa pipeline-translator-hpa + +# 수동 스케일링 (필요시) +kubectl -n site11-pipeline scale deployment pipeline-translator --replicas=5 +``` + +### K8s 워커 관리 + +#### 로그 확인 +```bash +# 특정 디플로이먼트 로그 +kubectl -n site11-pipeline logs -f deployment/pipeline-translator + +# 모든 Pod 로그 +kubectl -n site11-pipeline logs -f -l component=processor + +# 에러만 필터링 +kubectl -n site11-pipeline logs -f deployment/pipeline-translator | grep ERROR +``` + +#### 업데이트 및 롤백 +```bash +# 이미지 업데이트 +kubectl -n site11-pipeline set image deployment/pipeline-translator \ + translator=site11/pipeline-translator:v2 + +# 롤아웃 상태 확인 +kubectl -n site11-pipeline rollout status deployment/pipeline-translator + +# 롤백 (필요시) +kubectl -n site11-pipeline rollout undo deployment/pipeline-translator +``` + +#### 리소스 모니터링 +```bash +# Pod 리소스 사용량 +kubectl -n site11-pipeline top pods + +# Node 리소스 사용량 +kubectl top nodes + +# 상세 리소스 정보 +kubectl -n site11-pipeline describe pod +``` + +### 문제 해결 (K8s) + +#### Pod가 시작되지 않을 때 +```bash +# Pod 상태 확인 +kubectl -n site11-pipeline describe pod + +# 이벤트 확인 +kubectl -n site11-pipeline get events --sort-by='.lastTimestamp' + +# 이미지 풀 문제 확인 +kubectl -n site11-pipeline get pods -o jsonpath='{.items[*].spec.containers[*].image}' +``` + +#### Redis/MongoDB 연결 실패 +```bash +# Docker 호스트 네트워크 확인 +# K8s Pod는 host.docker.internal 사용 +kubectl -n site11-pipeline exec -it -- ping host.docker.internal + +# 연결 테스트 +kubectl -n site11-pipeline exec -it -- redis-cli -h host.docker.internal ping +``` + +#### 전체 정리 +```bash +# K8s 리소스 삭제 +kubectl delete namespace site11-pipeline + +# Docker 정리 +docker-compose -f docker-compose-hybrid.yml down + +# 볼륨 정리 (주의!) +docker-compose -f docker-compose-hybrid.yml down -v +``` + ## 라이선스 (License) Copyright (c) 2024 Site11 Project. All rights reserved. \ No newline at end of file diff --git a/docker-compose-hybrid.yml b/docker-compose-hybrid.yml new file mode 100644 index 0000000..a10f748 --- /dev/null +++ b/docker-compose-hybrid.yml @@ -0,0 +1,194 @@ +# Docker Compose for Hybrid Deployment (with K8s) +# ================================================ +# 이 파일은 K8s와 함께 사용하는 하이브리드 배포용입니다. +# Pipeline 워커들은 K8s로 이동하고, 인프라와 중앙 제어만 Docker에 유지합니다. + +version: '3.8' + +services: + # ============ Infrastructure Services ============ + mongodb: + image: mongo:7.0 + container_name: ${COMPOSE_PROJECT_NAME}_mongodb + ports: + - "27017:27017" + volumes: + - ./data/mongodb:/data/db + - ./data/mongodb-config:/data/configdb + environment: + - MONGO_INITDB_DATABASE=site11_db + networks: + - site11_network + restart: unless-stopped + + redis: + image: redis:7-alpine + container_name: ${COMPOSE_PROJECT_NAME}_redis + ports: + - "6379:6379" + volumes: + - ./data/redis:/data + command: redis-server --appendonly yes + networks: + - site11_network + restart: unless-stopped + + zookeeper: + image: confluentinc/cp-zookeeper:7.5.0 + container_name: ${COMPOSE_PROJECT_NAME}_zookeeper + environment: + ZOOKEEPER_CLIENT_PORT: 2181 + ZOOKEEPER_TICK_TIME: 2000 + volumes: + - ./data/zookeeper/data:/var/lib/zookeeper/data + - ./data/zookeeper/logs:/var/lib/zookeeper/logs + networks: + - site11_network + restart: unless-stopped + + kafka: + image: confluentinc/cp-kafka:7.5.0 + container_name: ${COMPOSE_PROJECT_NAME}_kafka + depends_on: + - zookeeper + ports: + - "9092:9092" + - "9101:9101" + environment: + KAFKA_BROKER_ID: 1 + KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' + KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT + KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092 + KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 + KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 + KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 + KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 + KAFKA_JMX_PORT: 9101 + KAFKA_JMX_HOSTNAME: localhost + KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true' + volumes: + - ./data/kafka:/var/lib/kafka/data + networks: + - site11_network + restart: unless-stopped + + # ============ Central Control & Monitoring ============ + + # Pipeline Scheduler - 중앙 제어 (Docker에 유지) + pipeline-scheduler: + build: + context: ./services/pipeline + dockerfile: scheduler/Dockerfile + container_name: ${COMPOSE_PROJECT_NAME}_pipeline_scheduler + restart: unless-stopped + depends_on: + - redis + - mongodb + environment: + - REDIS_URL=redis://redis:6379 + - MONGODB_URL=mongodb://mongodb:27017 + - DB_NAME=ai_writer_db + - LOG_LEVEL=INFO + - SCHEDULER_INTERVAL=60 + volumes: + - ./services/pipeline/shared:/app/shared:ro + - ./services/pipeline/config:/app/config:ro + networks: + - site11_network + + # Pipeline Monitor - 모니터링 대시보드 (Docker에 유지) + pipeline-monitor: + build: + context: ./services/pipeline + dockerfile: monitor/Dockerfile + container_name: ${COMPOSE_PROJECT_NAME}_pipeline_monitor + restart: unless-stopped + depends_on: + - redis + - mongodb + ports: + - "8100:8000" + environment: + - REDIS_URL=redis://redis:6379 + - MONGODB_URL=mongodb://mongodb:27017 + - DB_NAME=ai_writer_db + - LOG_LEVEL=INFO + volumes: + - ./services/pipeline/shared:/app/shared:ro + networks: + - site11_network + + # Pipeline Language Sync - 언어 동기화 (Docker에 유지) + pipeline-language-sync: + build: + context: ./services/pipeline + dockerfile: translator/Dockerfile + container_name: ${COMPOSE_PROJECT_NAME}_pipeline_language_sync + restart: unless-stopped + depends_on: + - redis + - mongodb + env_file: + - ./services/pipeline/.env + environment: + - REDIS_URL=redis://redis:6379 + - MONGODB_URL=mongodb://mongodb:27017 + - DB_NAME=ai_writer_db + - LOG_LEVEL=INFO + - MODE=sync # sync 모드로 실행 + volumes: + - ./services/pipeline/shared:/app/shared:ro + networks: + - site11_network + command: ["python", "language_sync.py"] + + # ============ Other Essential Services ============ + + console-backend: + build: + context: ./console/backend + dockerfile: Dockerfile + container_name: ${COMPOSE_PROJECT_NAME}_console_backend + ports: + - "8011:8000" + environment: + - ENV=${ENV} + - MONGODB_URL=mongodb://mongodb:27017 + - REDIS_URL=redis://redis:6379 + - USERS_SERVICE_URL=http://users-backend:8000 + depends_on: + - mongodb + - redis + networks: + - site11_network + restart: unless-stopped + + console-frontend: + build: + context: ./console/frontend + dockerfile: Dockerfile + container_name: ${COMPOSE_PROJECT_NAME}_console_frontend + ports: + - "3000:80" + environment: + - VITE_API_URL=http://localhost:8011 + networks: + - site11_network + restart: unless-stopped + +networks: + site11_network: + driver: bridge + name: site11_network + +# ============ K8s로 이동한 서비스들 ============ +# 다음 서비스들은 K8s에서 실행됩니다: +# - pipeline-rss-collector +# - pipeline-google-search +# - pipeline-translator +# - pipeline-ai-article-generator +# - pipeline-image-generator +# +# K8s 배포 방법: +# cd k8s/pipeline +# ./deploy.sh \ No newline at end of file diff --git a/k8s/pipeline/ai-article-generator.yaml b/k8s/pipeline/ai-article-generator.yaml new file mode 100644 index 0000000..9d80e25 --- /dev/null +++ b/k8s/pipeline/ai-article-generator.yaml @@ -0,0 +1,78 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pipeline-ai-article-generator + namespace: site11-pipeline + labels: + app: pipeline-ai-article-generator + component: processor +spec: + replicas: 2 + selector: + matchLabels: + app: pipeline-ai-article-generator + template: + metadata: + labels: + app: pipeline-ai-article-generator + component: processor + spec: + containers: + - name: ai-article-generator + image: site11/pipeline-ai-article-generator:latest + imagePullPolicy: Always + envFrom: + - configMapRef: + name: pipeline-config + - secretRef: + name: pipeline-secrets + resources: + requests: + memory: "512Mi" + cpu: "200m" + limits: + memory: "1Gi" + cpu: "1000m" + livenessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 10 + periodSeconds: 10 + +--- +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: pipeline-ai-article-generator-hpa + namespace: site11-pipeline +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: pipeline-ai-article-generator + minReplicas: 1 + maxReplicas: 8 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 70 + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: 80 \ No newline at end of file diff --git a/k8s/pipeline/configmap.yaml b/k8s/pipeline/configmap.yaml new file mode 100644 index 0000000..b7a5114 --- /dev/null +++ b/k8s/pipeline/configmap.yaml @@ -0,0 +1,38 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: pipeline-config + namespace: site11-pipeline +data: + # Redis 연결 (Docker 호스트) + REDIS_URL: "redis://host.docker.internal:6379" + + # MongoDB 연결 (Docker 호스트) + MONGODB_URL: "mongodb://host.docker.internal:27017" + DB_NAME: "ai_writer_db" + + # 로깅 + LOG_LEVEL: "INFO" + + # 워커 설정 + WORKER_COUNT: "2" + BATCH_SIZE: "10" + + # 큐 설정 + RSS_ENQUEUE_DELAY: "1.0" + GOOGLE_SEARCH_DELAY: "2.0" + TRANSLATION_DELAY: "1.0" + +--- +apiVersion: v1 +kind: Secret +metadata: + name: pipeline-secrets + namespace: site11-pipeline +type: Opaque +stringData: + # API Keys (실제 값) + DEEPL_API_KEY: "3abbc796-2515-44a8-972d-22dcf27ab54a" + OPENAI_API_KEY: "sk-openai-api-key-here" # OpenAI API 키 필요 + CLAUDE_API_KEY: "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" + SERP_API_KEY: "serp-api-key-here" # SERP API 키 필요 \ No newline at end of file diff --git a/k8s/pipeline/deploy.sh b/k8s/pipeline/deploy.sh new file mode 100755 index 0000000..bacc324 --- /dev/null +++ b/k8s/pipeline/deploy.sh @@ -0,0 +1,96 @@ +#!/bin/bash + +# Site11 Pipeline K8s Deployment Script +# ====================================== + +set -e + +echo "🚀 Site11 Pipeline K8s Deployment" +echo "=================================" + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' # No Color + +# Check if kubectl is available +if ! command -v kubectl &> /dev/null; then + echo -e "${RED}❌ kubectl is not installed${NC}" + exit 1 +fi + +# Check K8s cluster connection +echo -n "Checking K8s cluster connection... " +if kubectl cluster-info &> /dev/null; then + echo -e "${GREEN}✓${NC}" +else + echo -e "${RED}✗ Cannot connect to K8s cluster${NC}" + exit 1 +fi + +# Step 1: Create namespace +echo "" +echo "1. Creating namespace..." +kubectl apply -f namespace.yaml + +# Step 2: Create ConfigMap and Secrets +echo "" +echo "2. Creating ConfigMap and Secrets..." +echo -e "${YELLOW}⚠️ Remember to update API keys in configmap.yaml${NC}" +kubectl apply -f configmap.yaml + +# Step 3: Build and push Docker images +echo "" +echo "3. Building Docker images..." +echo -e "${YELLOW}Note: This script assumes images are already built${NC}" +echo "To build images, run from project root:" +echo " docker-compose build pipeline-rss-collector" +echo " docker-compose build pipeline-google-search" +echo " docker-compose build pipeline-translator" +echo " docker-compose build pipeline-ai-article-generator" +echo " docker-compose build pipeline-image-generator" + +# Step 4: Tag and push images (if using local registry) +echo "" +echo "4. Tagging images for K8s..." +services=("rss-collector" "google-search" "translator" "ai-article-generator" "image-generator") + +for service in "${services[@]}"; do + echo "Tagging pipeline-$service..." + docker tag site11_pipeline-$service:latest site11/pipeline-$service:latest +done + +# Step 5: Deploy services +echo "" +echo "5. Deploying services to K8s..." + +for service in "${services[@]}"; do + echo "Deploying $service..." + kubectl apply -f $service.yaml +done + +# Step 6: Check deployment status +echo "" +echo "6. Checking deployment status..." +kubectl -n site11-pipeline get deployments + +echo "" +echo "7. Waiting for pods to be ready..." +kubectl -n site11-pipeline wait --for=condition=Ready pods --all --timeout=300s || true + +# Step 7: Show final status +echo "" +echo "✅ Deployment Complete!" +echo "" +echo "Current status:" +kubectl -n site11-pipeline get pods +echo "" +echo "To view logs:" +echo " kubectl -n site11-pipeline logs -f deployment/pipeline-translator" +echo "" +echo "To scale deployments:" +echo " kubectl -n site11-pipeline scale deployment pipeline-translator --replicas=5" +echo "" +echo "To delete all resources:" +echo " kubectl delete namespace site11-pipeline" \ No newline at end of file diff --git a/k8s/pipeline/google-search.yaml b/k8s/pipeline/google-search.yaml new file mode 100644 index 0000000..93e3555 --- /dev/null +++ b/k8s/pipeline/google-search.yaml @@ -0,0 +1,78 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pipeline-google-search + namespace: site11-pipeline + labels: + app: pipeline-google-search + component: data-collector +spec: + replicas: 2 + selector: + matchLabels: + app: pipeline-google-search + template: + metadata: + labels: + app: pipeline-google-search + component: data-collector + spec: + containers: + - name: google-search + image: site11/pipeline-google-search:latest + imagePullPolicy: Always + envFrom: + - configMapRef: + name: pipeline-config + - secretRef: + name: pipeline-secrets + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "512Mi" + cpu: "500m" + livenessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 10 + periodSeconds: 10 + +--- +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: pipeline-google-search-hpa + namespace: site11-pipeline +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: pipeline-google-search + minReplicas: 1 + maxReplicas: 5 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 70 + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: 80 \ No newline at end of file diff --git a/k8s/pipeline/image-generator.yaml b/k8s/pipeline/image-generator.yaml new file mode 100644 index 0000000..fe193ee --- /dev/null +++ b/k8s/pipeline/image-generator.yaml @@ -0,0 +1,78 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pipeline-image-generator + namespace: site11-pipeline + labels: + app: pipeline-image-generator + component: processor +spec: + replicas: 2 + selector: + matchLabels: + app: pipeline-image-generator + template: + metadata: + labels: + app: pipeline-image-generator + component: processor + spec: + containers: + - name: image-generator + image: site11/pipeline-image-generator:latest + imagePullPolicy: Always + envFrom: + - configMapRef: + name: pipeline-config + - secretRef: + name: pipeline-secrets + resources: + requests: + memory: "512Mi" + cpu: "200m" + limits: + memory: "1Gi" + cpu: "1000m" + livenessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 10 + periodSeconds: 10 + +--- +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: pipeline-image-generator-hpa + namespace: site11-pipeline +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: pipeline-image-generator + minReplicas: 1 + maxReplicas: 6 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 70 + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: 80 \ No newline at end of file diff --git a/k8s/pipeline/namespace.yaml b/k8s/pipeline/namespace.yaml new file mode 100644 index 0000000..cb16b28 --- /dev/null +++ b/k8s/pipeline/namespace.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: site11-pipeline + labels: + name: site11-pipeline + environment: production \ No newline at end of file diff --git a/k8s/pipeline/rss-collector.yaml b/k8s/pipeline/rss-collector.yaml new file mode 100644 index 0000000..9a1c38c --- /dev/null +++ b/k8s/pipeline/rss-collector.yaml @@ -0,0 +1,78 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pipeline-rss-collector + namespace: site11-pipeline + labels: + app: pipeline-rss-collector + component: data-collector +spec: + replicas: 2 + selector: + matchLabels: + app: pipeline-rss-collector + template: + metadata: + labels: + app: pipeline-rss-collector + component: data-collector + spec: + containers: + - name: rss-collector + image: site11/pipeline-rss-collector:latest + imagePullPolicy: Always + envFrom: + - configMapRef: + name: pipeline-config + - secretRef: + name: pipeline-secrets + resources: + requests: + memory: "256Mi" + cpu: "100m" + limits: + memory: "512Mi" + cpu: "500m" + livenessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 10 + periodSeconds: 10 + +--- +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: pipeline-rss-collector-hpa + namespace: site11-pipeline +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: pipeline-rss-collector + minReplicas: 1 + maxReplicas: 5 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 70 + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: 80 \ No newline at end of file diff --git a/k8s/pipeline/translator.yaml b/k8s/pipeline/translator.yaml new file mode 100644 index 0000000..cd67c5e --- /dev/null +++ b/k8s/pipeline/translator.yaml @@ -0,0 +1,78 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: pipeline-translator + namespace: site11-pipeline + labels: + app: pipeline-translator + component: processor +spec: + replicas: 3 + selector: + matchLabels: + app: pipeline-translator + template: + metadata: + labels: + app: pipeline-translator + component: processor + spec: + containers: + - name: translator + image: site11/pipeline-translator:latest + imagePullPolicy: Always + envFrom: + - configMapRef: + name: pipeline-config + - secretRef: + name: pipeline-secrets + resources: + requests: + memory: "512Mi" + cpu: "200m" + limits: + memory: "1Gi" + cpu: "1000m" + livenessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 30 + periodSeconds: 30 + readinessProbe: + exec: + command: + - python + - -c + - "import redis; r=redis.from_url('redis://host.docker.internal:6379'); r.ping()" + initialDelaySeconds: 10 + periodSeconds: 10 + +--- +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: pipeline-translator-hpa + namespace: site11-pipeline +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: pipeline-translator + minReplicas: 2 + maxReplicas: 10 + metrics: + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: 70 + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: 80 \ No newline at end of file