feat: Add MongoDB backups repository with comprehensive documentation
Backups included: - mongodb-20251022-093651.tar.gz (233 MB, 24 databases) - backup_20250928_214747.tar.gz (233 MB) Features: - Complete MongoDB dumps from site11_mongodb container - Compressed tar.gz format for efficient storage - Comprehensive README with restore procedures - Backup verification and automation guides - Emergency recovery documentation Database coverage: - ai_writer_db (461 MB) - Main content database - news_api_db - News API and outlets - Service databases (AI, Apple, Crypto, Google, Samsung, LG, WSJ, Musk, Korea) - Artist databases (NCT, TWICE, ENHYPEN, IVE, Stray Kids, BLACKPINK) - OAuth and pipeline databases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
6
.gitignore
vendored
Normal file
6
.gitignore
vendored
Normal file
@ -0,0 +1,6 @@
|
||||
# Ignore uncompressed backup directories
|
||||
mongodb-*/
|
||||
20*/
|
||||
|
||||
# Keep compressed backups only
|
||||
# *.tar.gz files will be tracked
|
||||
197
README.md
Normal file
197
README.md
Normal file
@ -0,0 +1,197 @@
|
||||
# Site11 Backups
|
||||
|
||||
MongoDB database backups for the Site11 project.
|
||||
|
||||
## Overview
|
||||
|
||||
This repository contains automated backups of all MongoDB databases used in the Site11 platform, including:
|
||||
- News API database
|
||||
- AI Writer database
|
||||
- Service-specific databases (AI, Apple, Crypto, Google, Samsung, etc.)
|
||||
- Artist databases (NCT, TWICE, ENHYPEN, IVE, Stray Kids, BLACKPINK)
|
||||
- OAuth and authentication databases
|
||||
|
||||
## Backup Schedule
|
||||
|
||||
Backups are created regularly and stored in compressed tar.gz format:
|
||||
- **Format**: `mongodb-YYYYMMDD-HHMMSS.tar.gz`
|
||||
- **Compression**: gzip
|
||||
- **Structure**: Full dump using `mongodump`
|
||||
|
||||
## Backup Contents
|
||||
|
||||
Each backup includes:
|
||||
- All databases and collections
|
||||
- Indexes
|
||||
- User data
|
||||
- Configuration data
|
||||
|
||||
### Latest Backup
|
||||
|
||||
- **Date**: 2025-10-22 09:36:51
|
||||
- **File**: `mongodb-20251022-093651.tar.gz`
|
||||
- **Size**: 233 MB (compressed), 768 MB (uncompressed)
|
||||
- **Databases**: 24 databases
|
||||
|
||||
## Restoring from Backup
|
||||
|
||||
### Extract the Backup
|
||||
|
||||
```bash
|
||||
# Extract the tar.gz file
|
||||
tar -xzf mongodb-YYYYMMDD-HHMMSS.tar.gz
|
||||
|
||||
# This will create a 'backup' directory with all database dumps
|
||||
cd mongodb-YYYYMMDD-HHMMSS/backup
|
||||
```
|
||||
|
||||
### Restore All Databases
|
||||
|
||||
```bash
|
||||
# Restore all databases
|
||||
mongorestore --host localhost --port 27017 backup/
|
||||
|
||||
# Or restore to Docker container
|
||||
docker exec -i site11_mongodb mongorestore --archive < backup.archive
|
||||
```
|
||||
|
||||
### Restore Specific Database
|
||||
|
||||
```bash
|
||||
# Restore only a specific database
|
||||
mongorestore --host localhost --port 27017 --db ai_writer_db backup/ai_writer_db
|
||||
|
||||
# For Docker
|
||||
docker cp backup/ai_writer_db site11_mongodb:/tmp/
|
||||
docker exec site11_mongodb mongorestore --db ai_writer_db /tmp/ai_writer_db
|
||||
```
|
||||
|
||||
### Restore with Drop (Replace existing data)
|
||||
|
||||
```bash
|
||||
# WARNING: This will drop existing collections before restoring
|
||||
mongorestore --host localhost --port 27017 --drop backup/
|
||||
```
|
||||
|
||||
## Backup Details
|
||||
|
||||
### Included Databases
|
||||
|
||||
| Database | Size | Description |
|
||||
|----------|------|-------------|
|
||||
| ai_writer_db | 461 MB | Main AI content generation database |
|
||||
| news_api_db | ~7 MB | News API data and outlets |
|
||||
| ai_service | ~0.18 MB | AI service configuration |
|
||||
| apple_service | ~0.18 MB | Apple news service |
|
||||
| crypto_service | ~0.18 MB | Cryptocurrency news |
|
||||
| google_service | ~0.18 MB | Google news service |
|
||||
| samsung_service | ~0.18 MB | Samsung news service |
|
||||
| lg_service | ~0.18 MB | LG news service |
|
||||
| wsj_service | ~0.18 MB | Wall Street Journal service |
|
||||
| musk_service | ~0.18 MB | Elon Musk news service |
|
||||
| korea_service | ~0.18 MB | Korea news service |
|
||||
| artist_*_service | ~0.18 MB each | K-pop artist news services |
|
||||
| oauth_db | ~0.05 MB | Authentication and OAuth data |
|
||||
| pipeline_db | ~0.32 MB | Data pipeline configuration |
|
||||
| k_culture_db | ~0.05 MB | Korean culture content |
|
||||
| files_db | ~0.03 MB | File storage metadata |
|
||||
|
||||
## Creating a New Backup
|
||||
|
||||
### Using Docker
|
||||
|
||||
```bash
|
||||
# Set backup directory
|
||||
BACKUP_DIR="backups/mongodb-$(date +%Y%m%d-%H%M%S)"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
|
||||
# Create backup
|
||||
docker exec site11_mongodb mongodump --out /tmp/backup --quiet
|
||||
docker cp site11_mongodb:/tmp/backup "$BACKUP_DIR/"
|
||||
docker exec site11_mongodb rm -rf /tmp/backup
|
||||
|
||||
# Compress
|
||||
cd backups
|
||||
tar -czf "$(basename "$BACKUP_DIR").tar.gz" "$(basename "$BACKUP_DIR")"
|
||||
|
||||
# Optional: Remove uncompressed backup
|
||||
rm -rf "$(basename "$BACKUP_DIR")"
|
||||
```
|
||||
|
||||
### Using MongoDB Tools
|
||||
|
||||
```bash
|
||||
# Direct backup
|
||||
mongodump --host localhost --port 27017 --out backup/$(date +%Y%m%d-%H%M%S)
|
||||
|
||||
# Compress
|
||||
tar -czf backup-$(date +%Y%m%d-%H%M%S).tar.gz backup/
|
||||
```
|
||||
|
||||
## Backup Verification
|
||||
|
||||
After creating a backup, verify its integrity:
|
||||
|
||||
```bash
|
||||
# Check tar.gz file integrity
|
||||
tar -tzf mongodb-YYYYMMDD-HHMMSS.tar.gz > /dev/null && echo "OK" || echo "CORRUPTED"
|
||||
|
||||
# Check file size
|
||||
ls -lh mongodb-YYYYMMDD-HHMMSS.tar.gz
|
||||
|
||||
# List databases in backup
|
||||
tar -xzf mongodb-YYYYMMDD-HHMMSS.tar.gz -O backup/ | head -20
|
||||
```
|
||||
|
||||
## Storage Recommendations
|
||||
|
||||
- Keep at least **3 recent backups** (daily)
|
||||
- Keep **weekly backups** for the last month
|
||||
- Keep **monthly backups** for the last year
|
||||
- Store critical backups in **multiple locations**:
|
||||
- Local storage
|
||||
- Git repository (this repo)
|
||||
- Cloud storage (S3, GCS, etc.)
|
||||
- External drive
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Backups contain **sensitive data** including user information
|
||||
- Ensure proper **access controls** on this repository
|
||||
- Consider **encryption** for production backups
|
||||
- Rotate backups regularly and **delete old backups** securely
|
||||
|
||||
## Automation
|
||||
|
||||
To automate backups, create a cron job:
|
||||
|
||||
```bash
|
||||
# Edit crontab
|
||||
crontab -e
|
||||
|
||||
# Add daily backup at 2 AM
|
||||
0 2 * * * cd /path/to/site11 && ./scripts/backup-mongodb.sh
|
||||
```
|
||||
|
||||
## Emergency Recovery
|
||||
|
||||
In case of data loss:
|
||||
|
||||
1. **Stop the application** to prevent data corruption
|
||||
2. **Identify the most recent valid backup**
|
||||
3. **Extract and verify** the backup
|
||||
4. **Restore the database** using mongorestore
|
||||
5. **Verify data integrity** after restoration
|
||||
6. **Restart the application**
|
||||
|
||||
## Support
|
||||
|
||||
For backup-related issues:
|
||||
1. Check MongoDB logs
|
||||
2. Verify disk space
|
||||
3. Test restore on a separate instance first
|
||||
4. Contact the development team if needed
|
||||
|
||||
## License
|
||||
|
||||
Proprietary - All rights reserved
|
||||
BIN
backup_20250928_214747.tar.gz
Normal file
BIN
backup_20250928_214747.tar.gz
Normal file
Binary file not shown.
BIN
mongodb-20251022-093651.tar.gz
Normal file
BIN
mongodb-20251022-093651.tar.gz
Normal file
Binary file not shown.
BIN
mongodb/.DS_Store
vendored
Normal file
BIN
mongodb/.DS_Store
vendored
Normal file
Binary file not shown.
Reference in New Issue
Block a user