jungwoo choi 070032006e feat: Implement async queue-based news pipeline with microservices
Major architectural transformation from synchronous to asynchronous processing:

## Pipeline Services (8 microservices)
- pipeline-scheduler: APScheduler for 30-minute periodic job triggers
- pipeline-rss-collector: RSS feed collection with deduplication (7-day TTL)
- pipeline-google-search: Content enrichment via Google Search API
- pipeline-ai-summarizer: AI summarization using Claude API (claude-sonnet-4-20250514)
- pipeline-translator: Translation using DeepL Pro API
- pipeline-image-generator: Image generation with Replicate API (Stable Diffusion)
- pipeline-article-assembly: Final article assembly and MongoDB storage
- pipeline-monitor: Real-time monitoring dashboard (port 8100)

## Key Features
- Redis-based job queue with deduplication
- Asynchronous processing with Python asyncio
- Shared models and queue manager for inter-service communication
- Docker containerization for all services
- Container names standardized with site11_ prefix

## Removed Services
- Moved to backup: google-search, rss-feed, news-aggregator, ai-writer

## Configuration
- DeepL Pro API: 3abbc796-2515-44a8-972d-22dcf27ab54a
- Claude Model: claude-sonnet-4-20250514
- Redis Queue TTL: 7 days for deduplication

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-13 19:22:14 +09:00

Site11 - Microservices Architecture

Overview

Microservices platform with Console as API Gateway orchestrating multiple domain services.

Quick Start

Start Services

# Start console service
docker-compose up -d console-backend

# Check status
curl http://localhost:8011/health

Available Endpoints

  • http://localhost:8011/ - Root endpoint
  • http://localhost:8011/health - Health check
  • http://localhost:8011/api/status - System status

Architecture

  • Console: API Gateway and orchestrator
  • Services: Domain-specific microservices (users, oauth, images, etc.)
  • Database: MongoDB for persistence
  • Cache: Redis for caching and pub/sub

Development

See docs/PLAN.md for implementation roadmap and docs/PROGRESS.md for current status.

Description
No description provided
Readme 1.1 MiB
Languages
Python 80.8%
TypeScript 13.3%
Shell 4.9%
Dockerfile 0.7%
Makefile 0.2%