commit 80d515fbe98c1b1f635e705f59b0e318c66f9e65 Author: jungwoo choi Date: Thu Oct 23 14:54:16 2025 +0900 feat: Add MongoDB backups repository with comprehensive documentation Backups included: - mongodb-20251022-093651.tar.gz (233 MB, 24 databases) - backup_20250928_214747.tar.gz (233 MB) Features: - Complete MongoDB dumps from site11_mongodb container - Compressed tar.gz format for efficient storage - Comprehensive README with restore procedures - Backup verification and automation guides - Emergency recovery documentation Database coverage: - ai_writer_db (461 MB) - Main content database - news_api_db - News API and outlets - Service databases (AI, Apple, Crypto, Google, Samsung, LG, WSJ, Musk, Korea) - Artist databases (NCT, TWICE, ENHYPEN, IVE, Stray Kids, BLACKPINK) - OAuth and pipeline databases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude diff --git a/.DS_Store b/.DS_Store new file mode 100644 index 0000000..613fdc4 Binary files /dev/null and b/.DS_Store differ diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..ccc2ed2 --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +# Ignore uncompressed backup directories +mongodb-*/ +20*/ + +# Keep compressed backups only +# *.tar.gz files will be tracked diff --git a/README.md b/README.md new file mode 100644 index 0000000..452474f --- /dev/null +++ b/README.md @@ -0,0 +1,197 @@ +# Site11 Backups + +MongoDB database backups for the Site11 project. + +## Overview + +This repository contains automated backups of all MongoDB databases used in the Site11 platform, including: +- News API database +- AI Writer database +- Service-specific databases (AI, Apple, Crypto, Google, Samsung, etc.) +- Artist databases (NCT, TWICE, ENHYPEN, IVE, Stray Kids, BLACKPINK) +- OAuth and authentication databases + +## Backup Schedule + +Backups are created regularly and stored in compressed tar.gz format: +- **Format**: `mongodb-YYYYMMDD-HHMMSS.tar.gz` +- **Compression**: gzip +- **Structure**: Full dump using `mongodump` + +## Backup Contents + +Each backup includes: +- All databases and collections +- Indexes +- User data +- Configuration data + +### Latest Backup + +- **Date**: 2025-10-22 09:36:51 +- **File**: `mongodb-20251022-093651.tar.gz` +- **Size**: 233 MB (compressed), 768 MB (uncompressed) +- **Databases**: 24 databases + +## Restoring from Backup + +### Extract the Backup + +```bash +# Extract the tar.gz file +tar -xzf mongodb-YYYYMMDD-HHMMSS.tar.gz + +# This will create a 'backup' directory with all database dumps +cd mongodb-YYYYMMDD-HHMMSS/backup +``` + +### Restore All Databases + +```bash +# Restore all databases +mongorestore --host localhost --port 27017 backup/ + +# Or restore to Docker container +docker exec -i site11_mongodb mongorestore --archive < backup.archive +``` + +### Restore Specific Database + +```bash +# Restore only a specific database +mongorestore --host localhost --port 27017 --db ai_writer_db backup/ai_writer_db + +# For Docker +docker cp backup/ai_writer_db site11_mongodb:/tmp/ +docker exec site11_mongodb mongorestore --db ai_writer_db /tmp/ai_writer_db +``` + +### Restore with Drop (Replace existing data) + +```bash +# WARNING: This will drop existing collections before restoring +mongorestore --host localhost --port 27017 --drop backup/ +``` + +## Backup Details + +### Included Databases + +| Database | Size | Description | +|----------|------|-------------| +| ai_writer_db | 461 MB | Main AI content generation database | +| news_api_db | ~7 MB | News API data and outlets | +| ai_service | ~0.18 MB | AI service configuration | +| apple_service | ~0.18 MB | Apple news service | +| crypto_service | ~0.18 MB | Cryptocurrency news | +| google_service | ~0.18 MB | Google news service | +| samsung_service | ~0.18 MB | Samsung news service | +| lg_service | ~0.18 MB | LG news service | +| wsj_service | ~0.18 MB | Wall Street Journal service | +| musk_service | ~0.18 MB | Elon Musk news service | +| korea_service | ~0.18 MB | Korea news service | +| artist_*_service | ~0.18 MB each | K-pop artist news services | +| oauth_db | ~0.05 MB | Authentication and OAuth data | +| pipeline_db | ~0.32 MB | Data pipeline configuration | +| k_culture_db | ~0.05 MB | Korean culture content | +| files_db | ~0.03 MB | File storage metadata | + +## Creating a New Backup + +### Using Docker + +```bash +# Set backup directory +BACKUP_DIR="backups/mongodb-$(date +%Y%m%d-%H%M%S)" +mkdir -p "$BACKUP_DIR" + +# Create backup +docker exec site11_mongodb mongodump --out /tmp/backup --quiet +docker cp site11_mongodb:/tmp/backup "$BACKUP_DIR/" +docker exec site11_mongodb rm -rf /tmp/backup + +# Compress +cd backups +tar -czf "$(basename "$BACKUP_DIR").tar.gz" "$(basename "$BACKUP_DIR")" + +# Optional: Remove uncompressed backup +rm -rf "$(basename "$BACKUP_DIR")" +``` + +### Using MongoDB Tools + +```bash +# Direct backup +mongodump --host localhost --port 27017 --out backup/$(date +%Y%m%d-%H%M%S) + +# Compress +tar -czf backup-$(date +%Y%m%d-%H%M%S).tar.gz backup/ +``` + +## Backup Verification + +After creating a backup, verify its integrity: + +```bash +# Check tar.gz file integrity +tar -tzf mongodb-YYYYMMDD-HHMMSS.tar.gz > /dev/null && echo "OK" || echo "CORRUPTED" + +# Check file size +ls -lh mongodb-YYYYMMDD-HHMMSS.tar.gz + +# List databases in backup +tar -xzf mongodb-YYYYMMDD-HHMMSS.tar.gz -O backup/ | head -20 +``` + +## Storage Recommendations + +- Keep at least **3 recent backups** (daily) +- Keep **weekly backups** for the last month +- Keep **monthly backups** for the last year +- Store critical backups in **multiple locations**: + - Local storage + - Git repository (this repo) + - Cloud storage (S3, GCS, etc.) + - External drive + +## Security Notes + +- Backups contain **sensitive data** including user information +- Ensure proper **access controls** on this repository +- Consider **encryption** for production backups +- Rotate backups regularly and **delete old backups** securely + +## Automation + +To automate backups, create a cron job: + +```bash +# Edit crontab +crontab -e + +# Add daily backup at 2 AM +0 2 * * * cd /path/to/site11 && ./scripts/backup-mongodb.sh +``` + +## Emergency Recovery + +In case of data loss: + +1. **Stop the application** to prevent data corruption +2. **Identify the most recent valid backup** +3. **Extract and verify** the backup +4. **Restore the database** using mongorestore +5. **Verify data integrity** after restoration +6. **Restart the application** + +## Support + +For backup-related issues: +1. Check MongoDB logs +2. Verify disk space +3. Test restore on a separate instance first +4. Contact the development team if needed + +## License + +Proprietary - All rights reserved diff --git a/backup_20250928_214747.tar.gz b/backup_20250928_214747.tar.gz new file mode 100644 index 0000000..1d0ef59 Binary files /dev/null and b/backup_20250928_214747.tar.gz differ diff --git a/mongodb-20251022-093651.tar.gz b/mongodb-20251022-093651.tar.gz new file mode 100644 index 0000000..6553c39 Binary files /dev/null and b/mongodb-20251022-093651.tar.gz differ diff --git a/mongodb/.DS_Store b/mongodb/.DS_Store new file mode 100644 index 0000000..5008ddf Binary files /dev/null and b/mongodb/.DS_Store differ