Backend Integration:
- Created PipelineClient for communicating with Pipeline Monitor (port 8100)
- Added proxy endpoints in monitoring.py:
* /api/v1/monitoring/pipeline/stats - Queue status and article counts
* /api/v1/monitoring/pipeline/health - Pipeline service health
* /api/v1/monitoring/pipeline/queues/{name} - Queue details
* /api/v1/monitoring/pipeline/workers - Worker status
Frontend Integration:
- Added Pipeline Monitor API functions to monitoring.ts
- Updated Monitoring page to display:
* Redis queue status (keyword, rss, search, summarize, assembly)
* Article statistics (today, total, active keywords)
* Pipeline health status
* Worker status for each pipeline type
Architecture:
- Console acts as API Gateway, proxying requests to Pipeline Monitor
- Pipeline Monitor (services/pipeline/monitor) manages:
* RSS Collector, Google Search, AI Summarizer, Article Assembly workers
* Redis queues for async job processing
* MongoDB for article and keyword storage
This integration allows the News Engine Console to monitor and display
real-time pipeline activity, queue status, and worker health.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Frontend Implementation:
- Applications page: OAuth app management with client ID/secret
* Client secret regeneration with secure display
* Redirect URI management with chip interface
* Copy-to-clipboard for credentials
- Articles page: News articles browser with filters
* Category, translation, and image status filters
* Article detail modal with full content
* Retry controls for failed translations/images
* Server-side pagination support
- Monitoring page: System health and metrics dashboard
* Real-time CPU, memory, and disk usage
* Database statistics display
* Services status monitoring
* Recent logs table with level filtering
* Auto-refresh toggle (30s interval)
All pages follow the established DataGrid + MainLayout pattern with:
- Consistent UI/UX across all management pages
- Material-UI components and styling
- Error handling and loading states
- Full API integration with backend endpoints
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Backend: Downgrade Pydantic from v2 to v1.10.13 for compatibility
- Backend: Fix ObjectId to string conversion in user service
- Backend: Update config to use pydantic BaseSettings (v1 import)
- Frontend: Downgrade ESLint packages for compatibility
- Frontend: Configure Vite proxy to use 127.0.0.1 instead of localhost
- Frontend: Set API client to use direct backend URL (127.0.0.1:8101)
- Frontend: Add package-lock.json for dependency locking
This resolves MongoDB connection issues and frontend-backend
communication problems caused by localhost resolving to IPv6.
Verified: Login and dashboard functionality working correctly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Frontend Setup:
- Vite + React 18 + TypeScript configuration
- Material-UI v7 integration
- React Query for data fetching
- Zustand for state management
- React Router for routing
Project Configuration:
- package.json with all dependencies (React, MUI, TanStack Query, Zustand, etc.)
- tsconfig.json with path aliases (@/components, @/pages, etc.)
- vite.config.ts with dev server and proxy settings
- Dockerfile and Dockerfile.dev for production and development
- nginx.conf for production deployment
- .env and .gitignore files
- docker-compose.yml for local development
TypeScript Types:
- Complete type definitions for all API models
- User, Keyword, Pipeline, Application types
- Monitoring and system status types
- API response and pagination types
API Client Implementation:
- axios client with interceptors
- Token refresh logic
- Error handling
- Auto token injection
- Complete API service functions:
* users.ts (11 endpoints)
* keywords.ts (8 endpoints)
* pipelines.ts (11 endpoints)
* applications.ts (7 endpoints)
* monitoring.ts (8 endpoints)
State Management:
- authStore with Zustand
- Login/logout functionality
- Token persistence
- Current user management
Pages Implemented:
- Login page with MUI components
- Dashboard page with basic layout
- App.tsx with protected routes
Docker Configuration:
- docker-compose.yml for backend + frontend
- Dockerfile for production build
- Dockerfile.dev for development hot reload
Files Created: 23 files
- Frontend structure: src/{api,pages,stores,types}
- Configuration files: 8 files
- Docker files: 3 files
Next Steps:
- Test frontend in Docker environment
- Implement sidebar navigation
- Create full management pages (Keywords, Pipelines, Users, etc.)
- Connect to backend API and test authentication
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit migrates all models to Pydantic v2 and adds comprehensive
testing infrastructure for the news-engine-console backend.
Model Changes (Pydantic v2 Migration):
- Removed PyObjectId custom validators (v1 pattern incompatible with v2)
- Changed all model id fields from Optional[PyObjectId] to Optional[str]
- Replaced class Config with model_config = ConfigDict(populate_by_name=True)
- Updated User, Keyword, Pipeline, and Application models
Service Changes (ObjectId Handling):
- Added ObjectId to string conversion in all service methods before creating model instances
- Updated UserService: get_users(), get_user_by_id(), get_user_by_username()
- Updated KeywordService: 6 methods with ObjectId conversions
- Updated PipelineService: 8 methods with ObjectId conversions
- Updated ApplicationService: 6 methods with ObjectId conversions
Testing Infrastructure:
- Created comprehensive test_api.py (700+ lines) with 8 test suites:
* Health check, Authentication, Users API, Keywords API, Pipelines API,
Applications API, Monitoring API
- Created test_motor.py for debugging Motor async MongoDB connection
- Added Dockerfile for containerized deployment
- Created fix_objectid.py helper script for automated ObjectId conversion
Configuration Updates:
- Changed backend port from 8100 to 8101 (avoid conflict with pipeline_monitor)
- Made get_database() async for proper FastAPI dependency injection
- Updated DB_NAME from ai_writer_db to news_engine_console_db
Bug Fixes:
- Fixed environment variable override issue (system env > .env file)
- Fixed Pydantic v2 validator incompatibility causing TypeError
- Fixed list comprehension in bulk_create_keywords to properly convert ObjectIds
Test Results:
- All 8 test suites passing (100% success rate)
- Tested 37 API endpoints across all services
- No validation errors or ObjectId conversion issues
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Phase 1 Backend 100% 완료:
✅ UserService (312 lines):
- 인증 시스템 (authenticate_user, JWT 토큰 생성)
- CRUD 전체 기능 (get, create, update, delete)
- 권한 기반 필터링 (role, disabled, search)
- 비밀번호 관리 (change_password, hash 검증)
- 상태 토글 및 통계 조회
✅ ApplicationService (254 lines):
- OAuth2 클라이언트 관리
- Client ID/Secret 자동 생성
- Secret 재생성 기능
- 소유권 검증 (ownership check)
- 통계 조회 (grant types별)
✅ MonitoringService (309 lines):
- 시스템 헬스 체크 (MongoDB, pipelines)
- 시스템 메트릭 (keywords, pipelines, users, apps)
- 활동 로그 조회 (필터링, 날짜 범위)
- 데이터베이스 통계 (크기, 컬렉션, 인덱스)
- 파이프라인 성능 분석
- 에러 요약
✅ Users API (11 endpoints + OAuth2 로그인):
- POST /login - OAuth2 password flow
- GET /me - 현재 사용자 정보
- GET / - 사용자 목록 (admin only)
- GET /stats - 사용자 통계 (admin only)
- GET /{id} - 사용자 조회 (자신 or admin)
- POST / - 사용자 생성 (admin only)
- PUT /{id} - 사용자 수정 (권한 검증)
- DELETE /{id} - 사용자 삭제 (admin only, 자기 삭제 방지)
- POST /{id}/toggle - 상태 토글 (admin only)
- POST /change-password - 비밀번호 변경
✅ Applications API (7 endpoints):
- GET / - 애플리케이션 목록 (admin: 전체, user: 자신 것만)
- GET /stats - 통계 (admin only)
- GET /{id} - 조회 (소유자 or admin)
- POST / - 생성 (client_secret 1회만 표시)
- PUT /{id} - 수정 (소유자 or admin)
- DELETE /{id} - 삭제 (소유자 or admin)
- POST /{id}/regenerate-secret - Secret 재생성
✅ Monitoring API (8 endpoints):
- GET /health - 시스템 헬스 상태
- GET /metrics - 시스템 메트릭
- GET /logs - 활동 로그 (필터링 지원)
- GET /database/stats - DB 통계 (admin only)
- GET /database/collections - 컬렉션 통계 (admin only)
- GET /pipelines/performance - 파이프라인 성능
- GET /errors/summary - 에러 요약
주요 특징:
- 🔐 역할 기반 접근 제어 (RBAC: admin/editor/viewer)
- 🔒 OAuth2 Password Flow 인증
- 🛡️ 소유권 검증 (자신의 리소스만 수정)
- 🚫 안전 장치 (자기 삭제 방지, 자기 비활성화 방지)
- 📊 종합적인 모니터링 및 통계
- 🔑 안전한 Secret 관리 (1회만 표시)
- ✅ 완전한 에러 핸들링
Backend API 총 45개 엔드포인트 완성!
- Keywords: 8
- Pipelines: 11
- Users: 11
- Applications: 7
- Monitoring: 8
다음 단계:
- Frontend 구현 (React + TypeScript + Material-UI)
- Docker & Kubernetes 배포
- Redis 통합
- 테스트 작성
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
