#!/bin/bash # Site11 Pipeline Docker Hub Deployment Script # ============================================= # Push images to Docker Hub and deploy to K8s set -e echo "πŸš€ Site11 Pipeline Docker Hub Deployment" echo "========================================" echo "" # Colors for output RED='\033[0;31m' GREEN='\033[0;32m' YELLOW='\033[1;33m' BLUE='\033[0;34m' NC='\033[0m' # No Color # Configuration DOCKER_HUB_USER="${DOCKER_HUB_USER:-your-dockerhub-username}" # Set your Docker Hub username IMAGE_TAG="${IMAGE_TAG:-latest}" if [ "$DOCKER_HUB_USER" = "your-dockerhub-username" ]; then echo -e "${RED}❌ Please set DOCKER_HUB_USER environment variable${NC}" echo "Example: export DOCKER_HUB_USER=myusername" exit 1 fi # Check prerequisites echo -e "${BLUE}Checking prerequisites...${NC}" # Check if docker is logged in echo -n " Docker Hub login... " if docker info 2>/dev/null | grep -q "Username: $DOCKER_HUB_USER"; then echo -e "${GREEN}βœ“${NC}" else echo -e "${YELLOW}Please login${NC}" docker login fi # Check if kubectl is available if ! command -v kubectl &> /dev/null; then echo -e "${RED}❌ kubectl is not installed${NC}" exit 1 fi # Check K8s cluster connection echo -n " K8s cluster connection... " if kubectl cluster-info &> /dev/null; then echo -e "${GREEN}βœ“${NC}" else echo -e "${RED}βœ— Cannot connect to K8s cluster${NC}" exit 1 fi # Services to deploy services=("rss-collector" "google-search" "translator" "ai-article-generator" "image-generator") # Step 1: Tag and push images to Docker Hub echo "" echo -e "${BLUE}1. Pushing images to Docker Hub...${NC}" for service in "${services[@]}"; do echo -n " Pushing pipeline-$service... " docker tag site11-pipeline-$service:latest $DOCKER_HUB_USER/site11-pipeline-$service:$IMAGE_TAG docker push $DOCKER_HUB_USER/site11-pipeline-$service:$IMAGE_TAG && echo -e "${GREEN}βœ“${NC}" done # Step 2: Create namespace echo "" echo -e "${BLUE}2. Creating K8s namespace...${NC}" kubectl apply -f namespace.yaml # Step 3: Create ConfigMap and Secrets echo "" echo -e "${BLUE}3. Configuring external service connections...${NC}" cat > configmap-dockerhub.yaml << 'EOF' apiVersion: v1 kind: ConfigMap metadata: name: pipeline-config namespace: site11-pipeline data: # External Redis - AWS ElastiCache simulation REDIS_URL: "redis://host.docker.internal:6379" # External MongoDB - AWS DocumentDB simulation MONGODB_URL: "mongodb://host.docker.internal:27017" DB_NAME: "ai_writer_db" # Logging LOG_LEVEL: "INFO" # Worker settings WORKER_COUNT: "2" BATCH_SIZE: "10" # Queue delays RSS_ENQUEUE_DELAY: "1.0" GOOGLE_SEARCH_DELAY: "2.0" TRANSLATION_DELAY: "1.0" --- apiVersion: v1 kind: Secret metadata: name: pipeline-secrets namespace: site11-pipeline type: Opaque stringData: DEEPL_API_KEY: "3abbc796-2515-44a8-972d-22dcf27ab54a" CLAUDE_API_KEY: "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" OPENAI_API_KEY: "sk-openai-api-key-here" # Replace with actual key SERP_API_KEY: "serp-api-key-here" # Replace with actual key EOF kubectl apply -f configmap-dockerhub.yaml # Step 4: Create deployments using Docker Hub images echo "" echo -e "${BLUE}4. Creating K8s deployments...${NC}" for service in "${services[@]}"; do cat > ${service}-dockerhub.yaml << EOF apiVersion: apps/v1 kind: Deployment metadata: name: pipeline-$service namespace: site11-pipeline labels: app: pipeline-$service spec: replicas: $([ "$service" = "translator" ] && echo "3" || echo "2") selector: matchLabels: app: pipeline-$service template: metadata: labels: app: pipeline-$service spec: containers: - name: $service image: $DOCKER_HUB_USER/site11-pipeline-$service:$IMAGE_TAG imagePullPolicy: Always # Always pull from Docker Hub envFrom: - configMapRef: name: pipeline-config - secretRef: name: pipeline-secrets resources: requests: memory: "256Mi" cpu: "100m" limits: memory: "512Mi" cpu: "500m" readinessProbe: exec: command: - python - -c - "import sys; sys.exit(0)" initialDelaySeconds: 10 periodSeconds: 5 livenessProbe: exec: command: - python - -c - "import sys; sys.exit(0)" initialDelaySeconds: 30 periodSeconds: 10 --- apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: pipeline-$service-hpa namespace: site11-pipeline spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: pipeline-$service minReplicas: $([ "$service" = "translator" ] && echo "3" || echo "2") maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80 EOF done # Step 5: Deploy services to K8s echo "" echo -e "${BLUE}5. Deploying workers to K8s...${NC}" for service in "${services[@]}"; do echo -n " Deploying $service... " kubectl apply -f ${service}-dockerhub.yaml && echo -e "${GREEN}βœ“${NC}" done # Step 6: Wait for deployments echo "" echo -e "${BLUE}6. Waiting for pods to be ready...${NC}" kubectl -n site11-pipeline wait --for=condition=Ready pods --all --timeout=180s 2>/dev/null || { echo -e "${YELLOW}⚠️ Some pods are still initializing...${NC}" } # Step 7: Show status echo "" echo -e "${GREEN}βœ… Deployment Complete!${NC}" echo "" echo -e "${BLUE}Deployment status:${NC}" kubectl -n site11-pipeline get deployments echo "" echo -e "${BLUE}Pod status:${NC}" kubectl -n site11-pipeline get pods echo "" echo -e "${BLUE}Images deployed:${NC}" for service in "${services[@]}"; do echo " $DOCKER_HUB_USER/site11-pipeline-$service:$IMAGE_TAG" done echo "" echo -e "${BLUE}Useful commands:${NC}" echo " View logs: kubectl -n site11-pipeline logs -f deployment/pipeline-translator" echo " Scale: kubectl -n site11-pipeline scale deployment pipeline-translator --replicas=5" echo " Check HPA: kubectl -n site11-pipeline get hpa" echo " Update image: kubectl -n site11-pipeline set image deployment/pipeline-translator translator=$DOCKER_HUB_USER/site11-pipeline-translator:new-tag" echo " Delete: kubectl delete namespace site11-pipeline" echo "" echo -e "${BLUE}Architecture:${NC}" echo " 🌐 Images: Docker Hub ($DOCKER_HUB_USER/*)" echo " πŸ“¦ Infrastructure: External (Docker Compose)" echo " ☸️ Workers: K8s cluster" echo " πŸŽ›οΈ Control: Docker Compose (Scheduler, Monitor)"