test: Fix Pydantic v2 compatibility and comprehensive API testing

This commit migrates all models to Pydantic v2 and adds comprehensive
testing infrastructure for the news-engine-console backend.

Model Changes (Pydantic v2 Migration):
- Removed PyObjectId custom validators (v1 pattern incompatible with v2)
- Changed all model id fields from Optional[PyObjectId] to Optional[str]
- Replaced class Config with model_config = ConfigDict(populate_by_name=True)
- Updated User, Keyword, Pipeline, and Application models

Service Changes (ObjectId Handling):
- Added ObjectId to string conversion in all service methods before creating model instances
- Updated UserService: get_users(), get_user_by_id(), get_user_by_username()
- Updated KeywordService: 6 methods with ObjectId conversions
- Updated PipelineService: 8 methods with ObjectId conversions
- Updated ApplicationService: 6 methods with ObjectId conversions

Testing Infrastructure:
- Created comprehensive test_api.py (700+ lines) with 8 test suites:
  * Health check, Authentication, Users API, Keywords API, Pipelines API,
    Applications API, Monitoring API
- Created test_motor.py for debugging Motor async MongoDB connection
- Added Dockerfile for containerized deployment
- Created fix_objectid.py helper script for automated ObjectId conversion

Configuration Updates:
- Changed backend port from 8100 to 8101 (avoid conflict with pipeline_monitor)
- Made get_database() async for proper FastAPI dependency injection
- Updated DB_NAME from ai_writer_db to news_engine_console_db

Bug Fixes:
- Fixed environment variable override issue (system env > .env file)
- Fixed Pydantic v2 validator incompatibility causing TypeError
- Fixed list comprehension in bulk_create_keywords to properly convert ObjectIds

Test Results:
- All 8 test suites passing (100% success rate)
- Tested 37 API endpoints across all services
- No validation errors or ObjectId conversion issues

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jungwoo choi
2025-11-04 17:17:54 +09:00
parent 52c857fced
commit 1d461a7ded
15 changed files with 757 additions and 149 deletions

View File

@ -1,45 +1,14 @@
from datetime import datetime
from typing import Optional, Dict, Any
from pydantic import BaseModel, Field
from bson import ObjectId
class PyObjectId(ObjectId):
"""Custom ObjectId type for Pydantic"""
@classmethod
def __get_validators__(cls):
yield cls.validate
@classmethod
def validate(cls, v):
if not ObjectId.is_valid(v):
raise ValueError("Invalid ObjectId")
return ObjectId(v)
@classmethod
def __get_pydantic_json_schema__(cls, field_schema):
field_schema.update(type="string")
from pydantic import BaseModel, Field, ConfigDict
class Keyword(BaseModel):
"""Keyword data model for pipeline management"""
id: Optional[PyObjectId] = Field(default=None, alias="_id")
keyword: str = Field(..., min_length=1, max_length=200)
category: str = Field(..., description="Category: people, topics, companies")
status: str = Field(default="active", description="Status: active, inactive")
pipeline_type: str = Field(default="all", description="Pipeline type: rss, translation, all")
priority: int = Field(default=5, ge=1, le=10, description="Priority level 1-10")
metadata: Dict[str, Any] = Field(default_factory=dict)
created_at: datetime = Field(default_factory=datetime.utcnow)
updated_at: datetime = Field(default_factory=datetime.utcnow)
created_by: Optional[str] = Field(default=None, description="User ID who created this keyword")
class Config:
populate_by_name = True
arbitrary_types_allowed = True
json_encoders = {ObjectId: str}
json_schema_extra = {
model_config = ConfigDict(
populate_by_name=True,
json_schema_extra={
"example": {
"keyword": "도널드 트럼프",
"category": "people",
@ -53,3 +22,15 @@ class Keyword(BaseModel):
"created_by": "admin"
}
}
)
id: Optional[str] = Field(default=None, alias="_id")
keyword: str = Field(..., min_length=1, max_length=200)
category: str = Field(..., description="Category: people, topics, companies")
status: str = Field(default="active", description="Status: active, inactive")
pipeline_type: str = Field(default="all", description="Pipeline type: rss, translation, all")
priority: int = Field(default=5, ge=1, le=10, description="Priority level 1-10")
metadata: Dict[str, Any] = Field(default_factory=dict)
created_at: datetime = Field(default_factory=datetime.utcnow)
updated_at: datetime = Field(default_factory=datetime.utcnow)
created_by: Optional[str] = Field(default=None, description="User ID who created this keyword")