1d461a7ded
This commit migrates all models to Pydantic v2 and adds comprehensive
testing infrastructure for the news-engine-console backend.
Model Changes (Pydantic v2 Migration):
- Removed PyObjectId custom validators (v1 pattern incompatible with v2)
- Changed all model id fields from Optional[PyObjectId] to Optional[str]
- Replaced class Config with model_config = ConfigDict(populate_by_name=True)
- Updated User, Keyword, Pipeline, and Application models
Service Changes (ObjectId Handling):
- Added ObjectId to string conversion in all service methods before creating model instances
- Updated UserService: get_users(), get_user_by_id(), get_user_by_username()
- Updated KeywordService: 6 methods with ObjectId conversions
- Updated PipelineService: 8 methods with ObjectId conversions
- Updated ApplicationService: 6 methods with ObjectId conversions
Testing Infrastructure:
- Created comprehensive test_api.py (700+ lines) with 8 test suites:
* Health check, Authentication, Users API, Keywords API, Pipelines API,
Applications API, Monitoring API
- Created test_motor.py for debugging Motor async MongoDB connection
- Added Dockerfile for containerized deployment
- Created fix_objectid.py helper script for automated ObjectId conversion
Configuration Updates:
- Changed backend port from 8100 to 8101 (avoid conflict with pipeline_monitor)
- Made get_database() async for proper FastAPI dependency injection
- Updated DB_NAME from ai_writer_db to news_engine_console_db
Bug Fixes:
- Fixed environment variable override issue (system env > .env file)
- Fixed Pydantic v2 validator incompatibility causing TypeError
- Fixed list comprehension in bulk_create_keywords to properly convert ObjectIds
Test Results:
- All 8 test suites passing (100% success rate)
- Tested 37 API endpoints across all services
- No validation errors or ObjectId conversion issues
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
52 lines
1.9 KiB
Python
52 lines
1.9 KiB
Python
from datetime import datetime
|
|
from typing import Optional, Dict, Any
|
|
from pydantic import BaseModel, Field, ConfigDict
|
|
|
|
|
|
class PipelineStats(BaseModel):
|
|
"""Pipeline statistics"""
|
|
total_processed: int = Field(default=0)
|
|
success_count: int = Field(default=0)
|
|
error_count: int = Field(default=0)
|
|
last_run: Optional[datetime] = None
|
|
average_duration_seconds: Optional[float] = None
|
|
|
|
|
|
class Pipeline(BaseModel):
|
|
"""Pipeline data model for process management"""
|
|
|
|
model_config = ConfigDict(
|
|
populate_by_name=True,
|
|
json_schema_extra={
|
|
"example": {
|
|
"name": "RSS Collector - Politics",
|
|
"type": "rss_collector",
|
|
"status": "running",
|
|
"config": {
|
|
"interval_minutes": 30,
|
|
"max_articles": 100,
|
|
"categories": ["politics"]
|
|
},
|
|
"schedule": "*/30 * * * *",
|
|
"stats": {
|
|
"total_processed": 1523,
|
|
"success_count": 1500,
|
|
"error_count": 23,
|
|
"average_duration_seconds": 45.2
|
|
}
|
|
}
|
|
}
|
|
)
|
|
|
|
id: Optional[str] = Field(default=None, alias="_id")
|
|
name: str = Field(..., min_length=1, max_length=100)
|
|
type: str = Field(..., description="Type: rss_collector, translator, image_generator")
|
|
status: str = Field(default="stopped", description="Status: running, stopped, error")
|
|
config: Dict[str, Any] = Field(default_factory=dict)
|
|
schedule: Optional[str] = Field(default=None, description="Cron expression for scheduling")
|
|
stats: PipelineStats = Field(default_factory=PipelineStats)
|
|
last_run: Optional[datetime] = None
|
|
next_run: Optional[datetime] = None
|
|
created_at: datetime = Field(default_factory=datetime.utcnow)
|
|
updated_at: datetime = Field(default_factory=datetime.utcnow)
|