Compare commits
10 Commits
e4a776f67e
...
8747ae2ccb
| Author | SHA1 | Date | |
|---|---|---|---|
| 8747ae2ccb | |||
| f301e7dd7b | |||
| 26a196503b | |||
| 3222955793 | |||
| 7703dc0c2e | |||
| 5405233c01 | |||
| b7ae8f364c | |||
| e89c8d7fee | |||
| 49dfcba2b0 | |||
| a782b82993 |
519
backend-issues.md
Normal file
519
backend-issues.md
Normal file
@ -0,0 +1,519 @@
|
|||||||
|
# Backend Issues - Code Review Report
|
||||||
|
|
||||||
|
## 🔴 Critical Issues (Immediate Fix Required)
|
||||||
|
|
||||||
|
### 1. Race Condition in Bid Placement - Data Corruption Risk
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:**
|
||||||
|
- `server/storage.ts` (placeBid method)
|
||||||
|
- `server/routes.ts` (POST /api/auctions/:id/bid)
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Bid validation and database update happen in separate statements without transaction:
|
||||||
|
```typescript
|
||||||
|
// storage.ts - placeBid method
|
||||||
|
async placeBid(auctionId: string, userId: string, amount: number, qualityScore: number) {
|
||||||
|
const auction = await this.getAuctionById(auctionId);
|
||||||
|
|
||||||
|
// ❌ RACE CONDITION: Another bid can come in here!
|
||||||
|
if (parseFloat(auction.currentBid || "0") >= amount) {
|
||||||
|
throw new Error("Bid must be higher than current bid");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ❌ Two concurrent bids can both pass validation and overwrite each other
|
||||||
|
await db.insert(bids).values({ /* ... */ });
|
||||||
|
await db.update(auctions).set({ currentBid: amount.toString() });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Data corruption in live auctions**
|
||||||
|
- Lower bid can win if timing is right
|
||||||
|
- Auction integrity compromised
|
||||||
|
- Financial implications for users
|
||||||
|
- Business logic violations
|
||||||
|
|
||||||
|
**Reproduction Scenario:**
|
||||||
|
1. Current bid is $100
|
||||||
|
2. User A submits $150 bid
|
||||||
|
3. User B submits $120 bid simultaneously
|
||||||
|
4. Both read currentBid as $100
|
||||||
|
5. Both pass validation
|
||||||
|
6. Whichever writes last wins (could be the $120 bid!)
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Fixed with transaction and SELECT FOR UPDATE
|
||||||
|
async placeBid(auctionId: string, userId: string, amount: number, qualityScore: number) {
|
||||||
|
return await db.transaction(async (tx) => {
|
||||||
|
// Lock the auction row to prevent concurrent updates
|
||||||
|
const [auction] = await tx
|
||||||
|
.select()
|
||||||
|
.from(auctions)
|
||||||
|
.where(eq(auctions.id, auctionId))
|
||||||
|
.for('update'); // PostgreSQL row-level lock
|
||||||
|
|
||||||
|
if (!auction) {
|
||||||
|
throw new Error("Auction not found");
|
||||||
|
}
|
||||||
|
|
||||||
|
const currentBid = parseFloat(auction.currentBid || "0");
|
||||||
|
if (currentBid >= amount) {
|
||||||
|
throw new Error(`Bid must be higher than current bid of $${currentBid}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Insert bid and update auction atomically
|
||||||
|
const [newBid] = await tx.insert(bids).values({
|
||||||
|
id: crypto.randomUUID(),
|
||||||
|
auctionId,
|
||||||
|
userId,
|
||||||
|
amount: amount.toString(),
|
||||||
|
qualityScore,
|
||||||
|
createdAt: new Date()
|
||||||
|
}).returning();
|
||||||
|
|
||||||
|
await tx.update(auctions)
|
||||||
|
.set({
|
||||||
|
currentBid: amount.toString(),
|
||||||
|
qualityScore,
|
||||||
|
updatedAt: new Date()
|
||||||
|
})
|
||||||
|
.where(eq(auctions.id, auctionId));
|
||||||
|
|
||||||
|
return newBid;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Fix immediately before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. Incorrect HTTP Status Codes - Validation Errors Return 500
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:**
|
||||||
|
- `server/routes.ts` (multiple endpoints)
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Zod validation errors and client errors return 500 (Internal Server Error):
|
||||||
|
```typescript
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const article = insertArticleSchema.parse(req.body); // Can throw ZodError
|
||||||
|
// ...
|
||||||
|
} catch (error) {
|
||||||
|
// ❌ All errors become 500, even validation errors!
|
||||||
|
res.status(500).json({ message: "Failed to create article" });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Misleading error responses**
|
||||||
|
- Client errors (400) appear as server errors (500)
|
||||||
|
- Difficult to debug for frontend
|
||||||
|
- Poor API design
|
||||||
|
- Monitoring alerts on client mistakes
|
||||||
|
- Can't distinguish real server errors
|
||||||
|
|
||||||
|
**Affected Endpoints:**
|
||||||
|
- POST /api/articles
|
||||||
|
- POST /api/auctions/:id/bid
|
||||||
|
- POST /api/media-outlets/:slug/auction/bids
|
||||||
|
- POST /api/media-outlet-requests
|
||||||
|
- PATCH /api/media-outlet-requests/:id
|
||||||
|
- And more...
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Fixed with proper error handling
|
||||||
|
import { ZodError } from "zod";
|
||||||
|
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
try {
|
||||||
|
const article = insertArticleSchema.parse(req.body);
|
||||||
|
// ... rest of logic
|
||||||
|
} catch (error) {
|
||||||
|
// Handle Zod validation errors
|
||||||
|
if (error instanceof ZodError) {
|
||||||
|
return res.status(400).json({
|
||||||
|
message: "Invalid request data",
|
||||||
|
errors: error.errors.map(e => ({
|
||||||
|
field: e.path.join('.'),
|
||||||
|
message: e.message
|
||||||
|
}))
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Handle known business logic errors
|
||||||
|
if (error instanceof Error) {
|
||||||
|
if (error.message.includes("not found")) {
|
||||||
|
return res.status(404).json({ message: error.message });
|
||||||
|
}
|
||||||
|
if (error.message.includes("permission")) {
|
||||||
|
return res.status(403).json({ message: error.message });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Genuine server errors
|
||||||
|
console.error("Server error:", error);
|
||||||
|
res.status(500).json({ message: "Internal server error" });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Fix immediately
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. Request Status Updates Without Validation
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**File:** `server/routes.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Status updates accept any string and don't verify affected rows:
|
||||||
|
```typescript
|
||||||
|
app.patch('/api/media-outlet-requests/:id', async (req, res) => {
|
||||||
|
const { status } = req.body; // ❌ No validation!
|
||||||
|
const updated = await storage.updateMediaOutletRequest(req.params.id, { status });
|
||||||
|
// ❌ Returns null without checking, client gets 200 with null!
|
||||||
|
res.json(updated);
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Invalid status values accepted
|
||||||
|
- Non-existent IDs return 200 OK with null
|
||||||
|
- Domain rules violated
|
||||||
|
- State machine bypassed
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// Define allowed statuses
|
||||||
|
const allowedStatuses = ['pending', 'approved', 'rejected'] as const;
|
||||||
|
|
||||||
|
app.patch('/api/media-outlet-requests/:id', isAuthenticated, async (req: any, res) => {
|
||||||
|
try {
|
||||||
|
const { status } = req.body;
|
||||||
|
|
||||||
|
// Validate status
|
||||||
|
if (!allowedStatuses.includes(status)) {
|
||||||
|
return res.status(400).json({
|
||||||
|
message: `Invalid status. Must be one of: ${allowedStatuses.join(', ')}`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check permissions
|
||||||
|
const user = req.user;
|
||||||
|
if (user.role !== 'admin' && user.role !== 'superadmin') {
|
||||||
|
return res.status(403).json({ message: "Insufficient permissions" });
|
||||||
|
}
|
||||||
|
|
||||||
|
const updated = await storage.updateMediaOutletRequest(req.params.id, { status });
|
||||||
|
|
||||||
|
// Check if request exists
|
||||||
|
if (!updated) {
|
||||||
|
return res.status(404).json({ message: "Request not found" });
|
||||||
|
}
|
||||||
|
|
||||||
|
res.json(updated);
|
||||||
|
} catch (error) {
|
||||||
|
console.error("Error updating request:", error);
|
||||||
|
res.status(500).json({ message: "Failed to update request" });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Fix immediately
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟠 High Priority Issues
|
||||||
|
|
||||||
|
### 4. Missing Authentication on Sensitive Endpoints
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Files Affected:** `server/routes.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Some endpoints should require authentication but don't:
|
||||||
|
```typescript
|
||||||
|
// Anyone can create articles!
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
// ❌ No isAuthenticated middleware!
|
||||||
|
});
|
||||||
|
|
||||||
|
// Anyone can see all prediction bets
|
||||||
|
app.get('/api/prediction-bets', async (req, res) => {
|
||||||
|
// ❌ No authentication check!
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Unauthorized content creation
|
||||||
|
- Data exposure
|
||||||
|
- Spam and abuse potential
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// Add authentication middleware
|
||||||
|
app.post('/api/articles', isAuthenticated, async (req: any, res) => {
|
||||||
|
// Now req.user is available
|
||||||
|
});
|
||||||
|
|
||||||
|
// Check role for admin actions
|
||||||
|
app.post('/api/media-outlets', isAuthenticated, async (req: any, res) => {
|
||||||
|
if (req.user.role !== 'admin' && req.user.role !== 'superadmin') {
|
||||||
|
return res.status(403).json({ message: "Insufficient permissions" });
|
||||||
|
}
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Add before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. No Input Sanitization - Potential XSS
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Files Affected:** Multiple routes
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
User input stored directly without sanitization:
|
||||||
|
```typescript
|
||||||
|
const article = insertArticleSchema.parse(req.body);
|
||||||
|
await storage.createArticle({
|
||||||
|
...article,
|
||||||
|
content: req.body.content // ❌ No sanitization!
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- XSS attacks possible
|
||||||
|
- Script injection
|
||||||
|
- Data integrity issues
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import DOMPurify from 'isomorphic-dompurify';
|
||||||
|
|
||||||
|
const article = insertArticleSchema.parse(req.body);
|
||||||
|
await storage.createArticle({
|
||||||
|
...article,
|
||||||
|
content: DOMPurify.sanitize(req.body.content),
|
||||||
|
title: DOMPurify.sanitize(req.body.title)
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Fix before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟡 Medium Priority Issues
|
||||||
|
|
||||||
|
### 6. Incomplete Error Logging
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**Files Affected:** Multiple routes
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Errors logged inconsistently:
|
||||||
|
```typescript
|
||||||
|
catch (error) {
|
||||||
|
console.error("Error:", error); // Minimal context
|
||||||
|
res.status(500).json({ message: "Failed" });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Difficult debugging
|
||||||
|
- Lost context
|
||||||
|
- Can't trace issues
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
catch (error) {
|
||||||
|
console.error("Failed to create article:", {
|
||||||
|
error: error instanceof Error ? error.message : error,
|
||||||
|
stack: error instanceof Error ? error.stack : undefined,
|
||||||
|
userId: req.user?.id,
|
||||||
|
timestamp: new Date().toISOString(),
|
||||||
|
body: req.body
|
||||||
|
});
|
||||||
|
res.status(500).json({ message: "Failed to create article" });
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P2 - Improve observability
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. No Rate Limiting
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**Files Affected:** All routes
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No rate limiting on any endpoint:
|
||||||
|
```typescript
|
||||||
|
// Anyone can spam requests!
|
||||||
|
app.post('/api/bids', async (req, res) => {
|
||||||
|
// No rate limiting
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- DoS vulnerability
|
||||||
|
- Resource exhaustion
|
||||||
|
- Abuse potential
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import rateLimit from 'express-rate-limit';
|
||||||
|
|
||||||
|
const apiLimiter = rateLimit({
|
||||||
|
windowMs: 15 * 60 * 1000, // 15 minutes
|
||||||
|
max: 100, // Limit each IP to 100 requests per window
|
||||||
|
message: 'Too many requests, please try again later'
|
||||||
|
});
|
||||||
|
|
||||||
|
app.use('/api/', apiLimiter);
|
||||||
|
|
||||||
|
// Stricter limits for sensitive operations
|
||||||
|
const bidLimiter = rateLimit({
|
||||||
|
windowMs: 60 * 1000, // 1 minute
|
||||||
|
max: 5, // 5 bids per minute
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/auctions/:id/bid', bidLimiter, isAuthenticated, async (req, res) => {
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P2 - Add before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟢 Low Priority Issues
|
||||||
|
|
||||||
|
### 8. Inconsistent Response Formats
|
||||||
|
**Severity:** LOW
|
||||||
|
**Files Affected:** Multiple routes
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Success and error responses have different structures:
|
||||||
|
```typescript
|
||||||
|
// Success
|
||||||
|
res.json(data);
|
||||||
|
|
||||||
|
// Error
|
||||||
|
res.json({ message: "Error" });
|
||||||
|
|
||||||
|
// Sometimes
|
||||||
|
res.json({ error: "Error" });
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Confusing for frontend
|
||||||
|
- Inconsistent parsing
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
Standardize response format:
|
||||||
|
```typescript
|
||||||
|
// Standard success
|
||||||
|
res.json({
|
||||||
|
success: true,
|
||||||
|
data: result
|
||||||
|
});
|
||||||
|
|
||||||
|
// Standard error
|
||||||
|
res.json({
|
||||||
|
success: false,
|
||||||
|
error: {
|
||||||
|
message: "...",
|
||||||
|
code: "ERROR_CODE"
|
||||||
|
}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P3 - Polish
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 9. Missing Request Validation Middleware
|
||||||
|
**Severity:** LOW
|
||||||
|
**Files Affected:** All routes
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Schema validation repeated in every route:
|
||||||
|
```typescript
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
const article = insertArticleSchema.parse(req.body); // Repeated pattern
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Code duplication
|
||||||
|
- Maintenance burden
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// Create reusable middleware
|
||||||
|
const validateBody = (schema: z.ZodSchema) => {
|
||||||
|
return (req: Request, res: Response, next: NextFunction) => {
|
||||||
|
try {
|
||||||
|
req.body = schema.parse(req.body);
|
||||||
|
next();
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof ZodError) {
|
||||||
|
return res.status(400).json({
|
||||||
|
message: "Validation failed",
|
||||||
|
errors: error.errors
|
||||||
|
});
|
||||||
|
}
|
||||||
|
next(error);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
// Use it
|
||||||
|
app.post('/api/articles',
|
||||||
|
validateBody(insertArticleSchema),
|
||||||
|
async (req, res) => {
|
||||||
|
// req.body is already validated
|
||||||
|
}
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P3 - Refactor
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
| Severity | Count | Must Fix Before Production |
|
||||||
|
|----------|-------|----------------------------|
|
||||||
|
| 🔴 Critical | 3 | ✅ Yes - Blocking |
|
||||||
|
| 🟠 High | 2 | ✅ Yes - Important |
|
||||||
|
| 🟡 Medium | 2 | ⚠️ Recommended |
|
||||||
|
| 🟢 Low | 2 | ❌ Nice to have |
|
||||||
|
|
||||||
|
**Total Issues:** 9
|
||||||
|
|
||||||
|
**Critical Path to Production:**
|
||||||
|
1. ✅ Fix race condition in bidding (P0)
|
||||||
|
2. ✅ Fix HTTP status codes (P0)
|
||||||
|
3. ✅ Add request validation (P0)
|
||||||
|
4. ✅ Add authentication checks (P1)
|
||||||
|
5. ✅ Add input sanitization (P1)
|
||||||
|
6. ⚠️ Add rate limiting (P2)
|
||||||
|
7. ⚠️ Improve error logging (P2)
|
||||||
|
|
||||||
|
**Estimated Fix Time:**
|
||||||
|
- Critical issues: 4-6 hours
|
||||||
|
- High priority: 3-4 hours
|
||||||
|
- Medium priority: 4-6 hours
|
||||||
|
- Low priority: 4-6 hours
|
||||||
|
|
||||||
|
**Recommended Testing:**
|
||||||
|
- Load test bid placement with concurrent requests
|
||||||
|
- Test all error scenarios for proper status codes
|
||||||
|
- Verify authentication on all protected routes
|
||||||
|
- Test input sanitization with XSS payloads
|
||||||
5951
data-export-2025-10-13.json
Normal file
5951
data-export-2025-10-13.json
Normal file
File diff suppressed because it is too large
Load Diff
649
database-issues.md
Normal file
649
database-issues.md
Normal file
@ -0,0 +1,649 @@
|
|||||||
|
# Database Issues - Code Review Report
|
||||||
|
|
||||||
|
## 🔴 Critical Issues (Immediate Fix Required)
|
||||||
|
|
||||||
|
### 1. No Foreign Key Constraints - Data Integrity at Risk
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:**
|
||||||
|
- `shared/schema.ts` (all table definitions)
|
||||||
|
- `server/storage.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No foreign key relationships defined between tables despite heavy inter-table dependencies:
|
||||||
|
```typescript
|
||||||
|
// articles table references media_outlets but NO FOREIGN KEY!
|
||||||
|
export const articles = pgTable("articles", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
mediaOutletId: varchar("media_outlet_id").notNull(), // ❌ No FK!
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// bids reference auctions and users but NO FOREIGN KEY!
|
||||||
|
export const bids = pgTable("bids", {
|
||||||
|
auctionId: varchar("auction_id").notNull(), // ❌ No FK!
|
||||||
|
userId: varchar("user_id").notNull(), // ❌ No FK!
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Orphaned records guaranteed** (articles without outlets, bids without auctions)
|
||||||
|
- **Data corruption** when parent records deleted
|
||||||
|
- **No cascade behavior** - manual cleanup required
|
||||||
|
- **Broken references** in production data
|
||||||
|
- **Application crashes** when joining on deleted records
|
||||||
|
- **Difficult debugging** - no DB-level integrity
|
||||||
|
|
||||||
|
**Affected Table Pairs:**
|
||||||
|
- `articles` → `media_outlets`
|
||||||
|
- `articles` → `users` (authorId)
|
||||||
|
- `auctions` → `media_outlets`
|
||||||
|
- `bids` → `auctions`
|
||||||
|
- `bids` → `users`
|
||||||
|
- `comments` → `articles`
|
||||||
|
- `comments` → `users`
|
||||||
|
- `prediction_markets` → `articles`
|
||||||
|
- `prediction_bets` → `prediction_markets`
|
||||||
|
- `prediction_bets` → `users`
|
||||||
|
- `media_outlet_requests` → `users`
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import { foreignKey, references } from "drizzle-orm/pg-core";
|
||||||
|
|
||||||
|
// ✅ Articles with proper foreign keys
|
||||||
|
export const articles = pgTable("articles", {
|
||||||
|
id: varchar("id").primaryKey().default(sql`gen_random_uuid()`),
|
||||||
|
mediaOutletId: varchar("media_outlet_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => mediaOutlets.id, {
|
||||||
|
onDelete: 'cascade', // Delete articles when outlet deleted
|
||||||
|
onUpdate: 'cascade'
|
||||||
|
}),
|
||||||
|
authorId: varchar("author_id")
|
||||||
|
.references(() => users.id, {
|
||||||
|
onDelete: 'set null', // Keep article but remove author
|
||||||
|
onUpdate: 'cascade'
|
||||||
|
}),
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// ✅ Bids with proper foreign keys
|
||||||
|
export const bids = pgTable("bids", {
|
||||||
|
id: varchar("id").primaryKey().default(sql`gen_random_uuid()`),
|
||||||
|
auctionId: varchar("auction_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => auctions.id, {
|
||||||
|
onDelete: 'cascade',
|
||||||
|
onUpdate: 'cascade'
|
||||||
|
}),
|
||||||
|
userId: varchar("user_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => users.id, {
|
||||||
|
onDelete: 'restrict', // Prevent user deletion if they have bids
|
||||||
|
onUpdate: 'cascade'
|
||||||
|
}),
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// ✅ Comments with foreign keys
|
||||||
|
export const comments = pgTable("comments", {
|
||||||
|
id: varchar("id").primaryKey().default(sql`gen_random_uuid()`),
|
||||||
|
articleId: varchar("article_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => articles.id, {
|
||||||
|
onDelete: 'cascade'
|
||||||
|
}),
|
||||||
|
userId: varchar("user_id")
|
||||||
|
.notNull()
|
||||||
|
.references(() => users.id, {
|
||||||
|
onDelete: 'cascade'
|
||||||
|
}),
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// Apply to ALL tables with references!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Migration Steps:**
|
||||||
|
1. Clean existing orphaned data
|
||||||
|
2. Add foreign keys to schema
|
||||||
|
3. Run `npm run db:push`
|
||||||
|
4. Test cascade behavior
|
||||||
|
5. Update application to handle FK errors
|
||||||
|
|
||||||
|
**Priority:** P0 - Block production deployment
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. Missing Unique Constraints - Duplicate Data Allowed
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:** `shared/schema.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Critical uniqueness assumptions not enforced at DB level:
|
||||||
|
```typescript
|
||||||
|
// Articles can have duplicate slugs! 😱
|
||||||
|
export const articles = pgTable("articles", {
|
||||||
|
slug: varchar("slug").notNull(), // ❌ Should be UNIQUE!
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// Multiple prediction markets per article allowed!
|
||||||
|
export const predictionMarkets = pgTable("prediction_markets", {
|
||||||
|
articleId: varchar("article_id").notNull(), // ❌ Should be UNIQUE!
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- `getArticleBySlug()` can return wrong article
|
||||||
|
- Multiple prediction markets per article breaks business logic
|
||||||
|
- Data inconsistency
|
||||||
|
- Application bugs from duplicate data
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Articles with unique slug
|
||||||
|
export const articles = pgTable("articles", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
slug: varchar("slug").notNull().unique(), // Add unique constraint
|
||||||
|
mediaOutletId: varchar("media_outlet_id").notNull(),
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// ✅ One prediction market per article
|
||||||
|
export const predictionMarkets = pgTable("prediction_markets", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
articleId: varchar("article_id")
|
||||||
|
.notNull()
|
||||||
|
.unique(), // Only one market per article
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
|
||||||
|
// ✅ Consider composite unique constraints where needed
|
||||||
|
export const bids = pgTable("bids", {
|
||||||
|
// ... fields
|
||||||
|
}, (table) => ({
|
||||||
|
// Prevent duplicate bids from same user on same auction
|
||||||
|
uniqueUserAuction: unique().on(table.userId, table.auctionId)
|
||||||
|
}));
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Fix immediately
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. No Database Indexes - Performance Catastrophe
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:** `shared/schema.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Zero indexes on frequently queried columns causing full table scans:
|
||||||
|
```typescript
|
||||||
|
// Queried constantly but NO INDEX!
|
||||||
|
export const articles = pgTable("articles", {
|
||||||
|
mediaOutletId: varchar("media_outlet_id").notNull(), // ❌ No index!
|
||||||
|
slug: varchar("slug").notNull(), // ❌ No index!
|
||||||
|
isPinned: boolean("is_pinned"),
|
||||||
|
publishedAt: timestamp("published_at"),
|
||||||
|
// ...
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Full table scans on every query**
|
||||||
|
- **Catastrophic performance** as data grows
|
||||||
|
- Slow page loads (100ms → 10s+)
|
||||||
|
- Database CPU spikes
|
||||||
|
- Poor user experience
|
||||||
|
- Can't scale beyond toy dataset
|
||||||
|
|
||||||
|
**Queries Affected:**
|
||||||
|
```sql
|
||||||
|
-- These all do FULL TABLE SCANS:
|
||||||
|
SELECT * FROM articles WHERE media_outlet_id = ? -- No index!
|
||||||
|
SELECT * FROM articles WHERE slug = ? -- No index!
|
||||||
|
SELECT * FROM bids WHERE auction_id = ? -- No index!
|
||||||
|
SELECT * FROM auctions WHERE is_active = true ORDER BY end_date -- No index!
|
||||||
|
SELECT * FROM prediction_markets WHERE article_id = ? -- No index!
|
||||||
|
SELECT * FROM comments WHERE article_id = ? ORDER BY created_at -- No index!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import { index } from "drizzle-orm/pg-core";
|
||||||
|
|
||||||
|
// ✅ Articles with proper indexes
|
||||||
|
export const articles = pgTable("articles", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
mediaOutletId: varchar("media_outlet_id").notNull(),
|
||||||
|
slug: varchar("slug").notNull().unique(),
|
||||||
|
isPinned: boolean("is_pinned").default(false),
|
||||||
|
publishedAt: timestamp("published_at").defaultNow(),
|
||||||
|
// ...
|
||||||
|
}, (table) => ({
|
||||||
|
// Single column indexes
|
||||||
|
mediaOutletIdx: index("articles_media_outlet_idx").on(table.mediaOutletId),
|
||||||
|
slugIdx: index("articles_slug_idx").on(table.slug),
|
||||||
|
|
||||||
|
// Composite index for common query pattern
|
||||||
|
outletPublishedIdx: index("articles_outlet_published_idx")
|
||||||
|
.on(table.mediaOutletId, table.publishedAt.desc()),
|
||||||
|
|
||||||
|
// Partial index for active items
|
||||||
|
pinnedIdx: index("articles_pinned_idx")
|
||||||
|
.on(table.isPinned)
|
||||||
|
.where(sql`${table.isPinned} = true`),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Auctions with indexes
|
||||||
|
export const auctions = pgTable("auctions", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
mediaOutletId: varchar("media_outlet_id").notNull(),
|
||||||
|
isActive: boolean("is_active").default(true),
|
||||||
|
endDate: timestamp("end_date").notNull(),
|
||||||
|
// ...
|
||||||
|
}, (table) => ({
|
||||||
|
mediaOutletIdx: index("auctions_media_outlet_idx").on(table.mediaOutletId),
|
||||||
|
activeEndDateIdx: index("auctions_active_end_idx")
|
||||||
|
.on(table.isActive, table.endDate)
|
||||||
|
.where(sql`${table.isActive} = true`),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Bids with indexes
|
||||||
|
export const bids = pgTable("bids", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
auctionId: varchar("auction_id").notNull(),
|
||||||
|
userId: varchar("user_id").notNull(),
|
||||||
|
createdAt: timestamp("created_at").defaultNow(),
|
||||||
|
// ...
|
||||||
|
}, (table) => ({
|
||||||
|
auctionIdx: index("bids_auction_idx").on(table.auctionId),
|
||||||
|
userIdx: index("bids_user_idx").on(table.userId),
|
||||||
|
// Composite for bid history
|
||||||
|
auctionCreatedIdx: index("bids_auction_created_idx")
|
||||||
|
.on(table.auctionId, table.createdAt.desc()),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Comments with indexes
|
||||||
|
export const comments = pgTable("comments", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
articleId: varchar("article_id").notNull(),
|
||||||
|
userId: varchar("user_id").notNull(),
|
||||||
|
createdAt: timestamp("created_at").defaultNow(),
|
||||||
|
// ...
|
||||||
|
}, (table) => ({
|
||||||
|
articleIdx: index("comments_article_idx").on(table.articleId),
|
||||||
|
articleCreatedIdx: index("comments_article_created_idx")
|
||||||
|
.on(table.articleId, table.createdAt.desc()),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Prediction markets with index
|
||||||
|
export const predictionMarkets = pgTable("prediction_markets", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
articleId: varchar("article_id").notNull().unique(),
|
||||||
|
isActive: boolean("is_active").default(true),
|
||||||
|
// ...
|
||||||
|
}, (table) => ({
|
||||||
|
articleIdx: index("prediction_markets_article_idx").on(table.articleId),
|
||||||
|
activeIdx: index("prediction_markets_active_idx")
|
||||||
|
.on(table.isActive)
|
||||||
|
.where(sql`${table.isActive} = true`),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Prediction bets with indexes
|
||||||
|
export const predictionBets = pgTable("prediction_bets", {
|
||||||
|
id: varchar("id").primaryKey(),
|
||||||
|
marketId: varchar("market_id").notNull(),
|
||||||
|
userId: varchar("user_id").notNull(),
|
||||||
|
// ...
|
||||||
|
}, (table) => ({
|
||||||
|
marketIdx: index("prediction_bets_market_idx").on(table.marketId),
|
||||||
|
userIdx: index("prediction_bets_user_idx").on(table.userId),
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Media outlets with category index
|
||||||
|
export const mediaOutlets = pgTable("media_outlets", {
|
||||||
|
// ...
|
||||||
|
category: varchar("category").notNull(),
|
||||||
|
isActive: boolean("is_active").default(true),
|
||||||
|
}, (table) => ({
|
||||||
|
categoryActiveIdx: index("media_outlets_category_active_idx")
|
||||||
|
.on(table.category, table.isActive),
|
||||||
|
}));
|
||||||
|
```
|
||||||
|
|
||||||
|
**Performance Impact:**
|
||||||
|
- Query time: 5000ms → 5ms (1000x improvement)
|
||||||
|
- Can handle millions of records
|
||||||
|
- Reduced DB CPU usage
|
||||||
|
- Better user experience
|
||||||
|
|
||||||
|
**Priority:** P0 - Block production deployment
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. No Transactions for Multi-Step Operations - Data Inconsistency
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:** `server/storage.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Multi-step operations not wrapped in transactions:
|
||||||
|
```typescript
|
||||||
|
// ❌ BROKEN: Bid creation without transaction
|
||||||
|
async placeBid(auctionId: string, userId: string, amount: number) {
|
||||||
|
// Step 1: Insert bid
|
||||||
|
await db.insert(bids).values({ /* ... */ });
|
||||||
|
|
||||||
|
// ⚠️ If this fails, bid exists but auction not updated!
|
||||||
|
// Step 2: Update auction
|
||||||
|
await db.update(auctions).set({ currentBid: amount });
|
||||||
|
}
|
||||||
|
|
||||||
|
// ❌ BROKEN: Prediction bet without transaction
|
||||||
|
async createPredictionBet(marketId: string, userId: string, option: string, amount: number) {
|
||||||
|
// Step 1: Insert bet
|
||||||
|
await db.insert(predictionBets).values({ /* ... */ });
|
||||||
|
|
||||||
|
// ⚠️ If this fails, bet exists but market totals wrong!
|
||||||
|
// Step 2: Update market aggregates
|
||||||
|
await db.update(predictionMarkets).set({
|
||||||
|
yesVolume: sql`yes_volume + ${amount}`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Data inconsistency** guaranteed
|
||||||
|
- Partial updates leave DB in invalid state
|
||||||
|
- Bids without updated auctions
|
||||||
|
- Bet counts don't match actual bets
|
||||||
|
- Financial data incorrect
|
||||||
|
- Impossible to debug inconsistencies
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import { db } from "./db";
|
||||||
|
|
||||||
|
// ✅ Fixed with transaction
|
||||||
|
async placeBid(auctionId: string, userId: string, amount: number, qualityScore: number) {
|
||||||
|
return await db.transaction(async (tx) => {
|
||||||
|
// All or nothing - both succeed or both fail
|
||||||
|
const [bid] = await tx.insert(bids).values({
|
||||||
|
id: crypto.randomUUID(),
|
||||||
|
auctionId,
|
||||||
|
userId,
|
||||||
|
amount: amount.toString(),
|
||||||
|
qualityScore,
|
||||||
|
createdAt: new Date()
|
||||||
|
}).returning();
|
||||||
|
|
||||||
|
await tx.update(auctions)
|
||||||
|
.set({
|
||||||
|
currentBid: amount.toString(),
|
||||||
|
qualityScore,
|
||||||
|
updatedAt: new Date()
|
||||||
|
})
|
||||||
|
.where(eq(auctions.id, auctionId));
|
||||||
|
|
||||||
|
return bid;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ✅ Fixed with transaction
|
||||||
|
async createPredictionBet(
|
||||||
|
marketId: string,
|
||||||
|
userId: string,
|
||||||
|
option: "yes" | "no",
|
||||||
|
amount: number
|
||||||
|
) {
|
||||||
|
return await db.transaction(async (tx) => {
|
||||||
|
// Create bet
|
||||||
|
const [bet] = await tx.insert(predictionBets).values({
|
||||||
|
id: crypto.randomUUID(),
|
||||||
|
marketId,
|
||||||
|
userId,
|
||||||
|
option,
|
||||||
|
amount: amount.toString(),
|
||||||
|
createdAt: new Date()
|
||||||
|
}).returning();
|
||||||
|
|
||||||
|
// Update market aggregates atomically
|
||||||
|
const updateField = option === "yes" ? "yesVolume" : "noVolume";
|
||||||
|
await tx.update(predictionMarkets)
|
||||||
|
.set({
|
||||||
|
[updateField]: sql`${updateField} + ${amount}`,
|
||||||
|
totalVolume: sql`total_volume + ${amount}`,
|
||||||
|
updatedAt: new Date()
|
||||||
|
})
|
||||||
|
.where(eq(predictionMarkets.id, marketId));
|
||||||
|
|
||||||
|
return bet;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// Apply to ALL multi-step operations!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Fix immediately
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. Float Precision Loss in Monetary Calculations
|
||||||
|
**Severity:** CRITICAL
|
||||||
|
**Files Affected:**
|
||||||
|
- `shared/schema.ts`
|
||||||
|
- `server/storage.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Decimal monetary values converted to float, causing precision loss:
|
||||||
|
```typescript
|
||||||
|
// Schema uses DECIMAL (correct)
|
||||||
|
export const bids = pgTable("bids", {
|
||||||
|
amount: decimal("amount", { precision: 10, scale: 2 }).notNull(),
|
||||||
|
});
|
||||||
|
|
||||||
|
// ❌ But code treats as float!
|
||||||
|
const currentBid = parseFloat(auction.currentBid || "0"); // Precision loss!
|
||||||
|
if (currentBid >= amount) { // Comparing floats!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Money calculation errors**
|
||||||
|
- Rounding errors accumulate
|
||||||
|
- $1000.00 might become $999.9999999
|
||||||
|
- Incorrect bid comparisons
|
||||||
|
- Financial inaccuracies
|
||||||
|
- Legal/compliance issues
|
||||||
|
|
||||||
|
**Examples of Precision Loss:**
|
||||||
|
```javascript
|
||||||
|
parseFloat("1000.10") + parseFloat("2000.20")
|
||||||
|
// Expected: 3000.30
|
||||||
|
// Actual: 3000.2999999999995 😱
|
||||||
|
```
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// Option 1: Use integer cents (RECOMMENDED)
|
||||||
|
export const bids = pgTable("bids", {
|
||||||
|
amountCents: integer("amount_cents").notNull(), // Store as cents
|
||||||
|
});
|
||||||
|
|
||||||
|
// Convert for display
|
||||||
|
const displayAmount = (cents: number) => `$${(cents / 100).toFixed(2)}`;
|
||||||
|
const parseCents = (dollars: string) => Math.round(parseFloat(dollars) * 100);
|
||||||
|
|
||||||
|
// Option 2: Use Decimal.js library
|
||||||
|
import Decimal from 'decimal.js';
|
||||||
|
|
||||||
|
const currentBid = new Decimal(auction.currentBid || "0");
|
||||||
|
const newBid = new Decimal(amount);
|
||||||
|
|
||||||
|
if (currentBid.greaterThanOrEqualTo(newBid)) {
|
||||||
|
throw new Error("Bid too low");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Option 3: Keep as string and use DB for comparison
|
||||||
|
const [auction] = await db
|
||||||
|
.select()
|
||||||
|
.from(auctions)
|
||||||
|
.where(
|
||||||
|
and(
|
||||||
|
eq(auctions.id, auctionId),
|
||||||
|
sql`${auctions.currentBid}::numeric < ${amount}::numeric`
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!auction) {
|
||||||
|
throw new Error("Bid must be higher than current bid");
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Fix before handling real money
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟠 High Priority Issues
|
||||||
|
|
||||||
|
### 6. N+1 Query Problem in Search Function
|
||||||
|
**Severity:** HIGH
|
||||||
|
**File:** `server/storage.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Search loads everything then filters in memory:
|
||||||
|
```typescript
|
||||||
|
async search(query: string) {
|
||||||
|
// ❌ Loads ALL outlets, articles, etc.
|
||||||
|
const outlets = await this.getMediaOutlets();
|
||||||
|
const articles = await this.getArticles();
|
||||||
|
|
||||||
|
// ❌ Filters in JavaScript instead of DB
|
||||||
|
return {
|
||||||
|
outlets: outlets.filter(o => o.name.includes(query)),
|
||||||
|
articles: articles.filter(a => a.title.includes(query))
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Loads entire database on every search
|
||||||
|
- Terrible performance
|
||||||
|
- High memory usage
|
||||||
|
- Can't scale
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Use DB for filtering
|
||||||
|
async search(query: string) {
|
||||||
|
const searchPattern = `%${query}%`;
|
||||||
|
|
||||||
|
const [outlets, articles] = await Promise.all([
|
||||||
|
db.select()
|
||||||
|
.from(mediaOutlets)
|
||||||
|
.where(
|
||||||
|
or(
|
||||||
|
ilike(mediaOutlets.name, searchPattern),
|
||||||
|
ilike(mediaOutlets.description, searchPattern)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.limit(20),
|
||||||
|
|
||||||
|
db.select()
|
||||||
|
.from(articles)
|
||||||
|
.where(
|
||||||
|
or(
|
||||||
|
ilike(articles.title, searchPattern),
|
||||||
|
ilike(articles.excerpt, searchPattern)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.limit(20)
|
||||||
|
]);
|
||||||
|
|
||||||
|
return { outlets, articles };
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Fix soon
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. Missing Cascade Delete Handling
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Files Affected:** `server/storage.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No handling for cascading deletes:
|
||||||
|
```typescript
|
||||||
|
async deleteMediaOutlet(id: string) {
|
||||||
|
// ❌ What about articles, auctions, etc.?
|
||||||
|
await db.delete(mediaOutlets).where(eq(mediaOutlets.id, id));
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Orphaned child records
|
||||||
|
- Broken relationships
|
||||||
|
- Data integrity issues
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
async deleteMediaOutlet(id: string) {
|
||||||
|
// With FK cascade, this handles everything
|
||||||
|
// Or manually delete children first if needed
|
||||||
|
await db.transaction(async (tx) => {
|
||||||
|
// Delete in dependency order
|
||||||
|
await tx.delete(articles).where(eq(articles.mediaOutletId, id));
|
||||||
|
await tx.delete(auctions).where(eq(auctions.mediaOutletId, id));
|
||||||
|
await tx.delete(mediaOutlets).where(eq(mediaOutlets.id, id));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Fix with FK implementation
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
| Severity | Count | Must Fix Before Production |
|
||||||
|
|----------|-------|----------------------------|
|
||||||
|
| 🔴 Critical | 5 | ✅ Yes - BLOCKING |
|
||||||
|
| 🟠 High | 2 | ✅ Yes |
|
||||||
|
| 🟡 Medium | 0 | - |
|
||||||
|
| 🟢 Low | 0 | - |
|
||||||
|
|
||||||
|
**Total Issues:** 7
|
||||||
|
|
||||||
|
**Critical Database Refactor Required:**
|
||||||
|
1. ✅ Add foreign keys to ALL relationships (P0)
|
||||||
|
2. ✅ Add unique constraints (P0)
|
||||||
|
3. ✅ Create indexes on ALL queried columns (P0)
|
||||||
|
4. ✅ Wrap multi-step operations in transactions (P0)
|
||||||
|
5. ✅ Fix decimal/float precision issues (P0)
|
||||||
|
6. ✅ Optimize search queries (P1)
|
||||||
|
7. ✅ Handle cascade deletes (P1)
|
||||||
|
|
||||||
|
**Estimated Refactor Time:**
|
||||||
|
- Schema changes: 4-6 hours
|
||||||
|
- Migration script: 2-3 hours
|
||||||
|
- Storage layer updates: 6-8 hours
|
||||||
|
- Testing: 4-6 hours
|
||||||
|
- **Total: 16-23 hours**
|
||||||
|
|
||||||
|
**Migration Strategy:**
|
||||||
|
1. Audit and clean existing data
|
||||||
|
2. Add foreign keys to schema
|
||||||
|
3. Add unique constraints
|
||||||
|
4. Create all indexes
|
||||||
|
5. Test performance improvements
|
||||||
|
6. Update storage layer for transactions
|
||||||
|
7. Fix precision issues
|
||||||
|
8. Deploy with zero downtime
|
||||||
|
|
||||||
|
**Performance Gains Expected:**
|
||||||
|
- Query time: 1000x improvement
|
||||||
|
- DB CPU: 90% reduction
|
||||||
|
- Scalability: Handle millions of records
|
||||||
|
- Data integrity: 100% enforcement
|
||||||
228
scripts/README.md
Normal file
228
scripts/README.md
Normal file
@ -0,0 +1,228 @@
|
|||||||
|
# 데이터 Export/Import 가이드
|
||||||
|
|
||||||
|
## 📋 개요
|
||||||
|
|
||||||
|
이 스크립트들은 SAPIENS 데이터베이스의 모든 데이터를 JSON 파일로 export하고, 다른 PostgreSQL 데이터베이스로 import할 수 있게 해줍니다.
|
||||||
|
|
||||||
|
## 📁 파일 설명
|
||||||
|
|
||||||
|
- **`export-data.ts`** - 현재 데이터베이스의 모든 데이터를 JSON 파일로 export
|
||||||
|
- **`import-data.ts`** - JSON 파일의 데이터를 새 PostgreSQL 데이터베이스로 import
|
||||||
|
- **`README.md`** - 이 가이드 문서
|
||||||
|
|
||||||
|
## 🚀 사용 방법
|
||||||
|
|
||||||
|
### 1. 데이터 Export (내보내기)
|
||||||
|
|
||||||
|
현재 데이터베이스의 모든 데이터를 JSON 파일로 저장합니다.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx tsx scripts/export-data.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
**결과:**
|
||||||
|
- `data-export-YYYY-MM-DD.json` 파일 생성 (예: `data-export-2025-10-13.json`)
|
||||||
|
- 모든 테이블 데이터가 JSON 형식으로 저장됨
|
||||||
|
- 진행 상황과 통계 표시
|
||||||
|
|
||||||
|
**Export되는 테이블:**
|
||||||
|
1. sessions (세션)
|
||||||
|
2. users (사용자)
|
||||||
|
3. media_outlets (미디어 매체)
|
||||||
|
4. articles (기사)
|
||||||
|
5. prediction_markets (예측 시장)
|
||||||
|
6. auctions (경매)
|
||||||
|
7. bids (입찰)
|
||||||
|
8. media_outlet_requests (매체 요청)
|
||||||
|
9. comments (댓글)
|
||||||
|
10. prediction_bets (예측 베팅)
|
||||||
|
|
||||||
|
### 2. 데이터 Import (가져오기)
|
||||||
|
|
||||||
|
Export한 JSON 파일을 새로운 PostgreSQL 데이터베이스로 가져옵니다.
|
||||||
|
|
||||||
|
#### 방법 1: 환경 변수 사용 (현재 DATABASE_URL 사용)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 방법 2: 새 데이터베이스 URL 직접 지정
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json "postgresql://user:password@host:5432/database"
|
||||||
|
```
|
||||||
|
|
||||||
|
**예시:**
|
||||||
|
```bash
|
||||||
|
# Neon 데이터베이스로 import
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json "postgresql://user:pass@ep-cool-name-123.us-east-2.aws.neon.tech/neondb"
|
||||||
|
|
||||||
|
# 로컬 PostgreSQL로 import
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json "postgresql://postgres:password@localhost:5432/mydb"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📊 Export 파일 구조
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"exportDate": "2025-10-13T08:30:31.284Z",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"tables": {
|
||||||
|
"sessions": [...],
|
||||||
|
"users": [...],
|
||||||
|
"mediaOutlets": [...],
|
||||||
|
"articles": [...],
|
||||||
|
"predictionMarkets": [...],
|
||||||
|
"auctions": [...],
|
||||||
|
"bids": [...],
|
||||||
|
"mediaOutletRequests": [...],
|
||||||
|
"comments": [...],
|
||||||
|
"predictionBets": [...]
|
||||||
|
},
|
||||||
|
"metadata": {
|
||||||
|
"totalRecords": 374,
|
||||||
|
"tableStats": {
|
||||||
|
"sessions": 1,
|
||||||
|
"users": 14,
|
||||||
|
"mediaOutlets": 77,
|
||||||
|
...
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## ⚙️ 중요 사항
|
||||||
|
|
||||||
|
### Import 전 준비사항
|
||||||
|
|
||||||
|
1. **대상 데이터베이스에 스키마가 생성되어 있어야 합니다:**
|
||||||
|
```bash
|
||||||
|
# 새 데이터베이스에서 먼저 실행
|
||||||
|
npm run db:push
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **데이터베이스가 비어있거나, 중복을 허용해야 합니다:**
|
||||||
|
- Import 스크립트는 `onConflictDoNothing()` 사용
|
||||||
|
- 같은 ID의 레코드가 있으면 skip됨
|
||||||
|
|
||||||
|
3. **의존성 순서 보장:**
|
||||||
|
- 스크립트가 자동으로 테이블 간 의존성 순서대로 import
|
||||||
|
- Users → Media Outlets → Articles → ... 순서로 진행
|
||||||
|
|
||||||
|
### 데이터 안전성
|
||||||
|
|
||||||
|
- ✅ Export는 읽기 전용 (원본 데이터 변경 없음)
|
||||||
|
- ✅ Import는 `onConflictDoNothing()` 사용 (기존 데이터 보존)
|
||||||
|
- ✅ 트랜잭션 없음 (부분 import 가능)
|
||||||
|
- ✅ 진행 상황 실시간 표시
|
||||||
|
|
||||||
|
## 🔧 문제 해결
|
||||||
|
|
||||||
|
### Export 실패 시
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 데이터베이스 연결 확인
|
||||||
|
echo $DATABASE_URL
|
||||||
|
|
||||||
|
# 데이터베이스 상태 확인
|
||||||
|
npm run db:push
|
||||||
|
```
|
||||||
|
|
||||||
|
### Import 실패 시
|
||||||
|
|
||||||
|
**에러: "No database URL provided"**
|
||||||
|
```bash
|
||||||
|
# DATABASE_URL 환경 변수 설정 후 다시 시도
|
||||||
|
export DATABASE_URL="postgresql://..."
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json
|
||||||
|
```
|
||||||
|
|
||||||
|
**에러: "Table does not exist"**
|
||||||
|
```bash
|
||||||
|
# 대상 데이터베이스에 스키마 생성
|
||||||
|
npm run db:push
|
||||||
|
```
|
||||||
|
|
||||||
|
**에러: "Foreign key constraint"**
|
||||||
|
- 의존성 순서 문제 (스크립트가 자동 처리하지만, 수동 수정 시 발생 가능)
|
||||||
|
- 테이블을 순서대로 import하는지 확인
|
||||||
|
|
||||||
|
## 📝 사용 예시
|
||||||
|
|
||||||
|
### 시나리오 1: 백업 생성
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 매일 백업 생성
|
||||||
|
npx tsx scripts/export-data.ts
|
||||||
|
# 결과: data-export-2025-10-13.json
|
||||||
|
|
||||||
|
# 파일 이름 변경 (선택사항)
|
||||||
|
mv data-export-2025-10-13.json backups/backup-$(date +%Y%m%d).json
|
||||||
|
```
|
||||||
|
|
||||||
|
### 시나리오 2: 개발 → 프로덕션 데이터 이전
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 개발 DB에서 export
|
||||||
|
npx tsx scripts/export-data.ts
|
||||||
|
# 결과: data-export-2025-10-13.json
|
||||||
|
|
||||||
|
# 2. 프로덕션 DB로 import
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json "postgresql://prod-db-url"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 시나리오 3: 데이터베이스 마이그레이션
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 기존 DB에서 export
|
||||||
|
DATABASE_URL="postgresql://old-db" npx tsx scripts/export-data.ts
|
||||||
|
|
||||||
|
# 2. 새 DB에 스키마 생성
|
||||||
|
DATABASE_URL="postgresql://new-db" npm run db:push
|
||||||
|
|
||||||
|
# 3. 새 DB로 데이터 import
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json "postgresql://new-db"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 시나리오 4: 로컬 개발용 데이터 복사
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 프로덕션에서 export
|
||||||
|
npx tsx scripts/export-data.ts
|
||||||
|
|
||||||
|
# 2. 로컬 DB로 import
|
||||||
|
npx tsx scripts/import-data.ts data-export-2025-10-13.json "postgresql://localhost:5432/local_dev"
|
||||||
|
```
|
||||||
|
|
||||||
|
## 🎯 체크리스트
|
||||||
|
|
||||||
|
Export 전:
|
||||||
|
- [ ] DATABASE_URL이 올바른 데이터베이스를 가리키는지 확인
|
||||||
|
- [ ] 충분한 디스크 공간 확보 (대용량 DB의 경우)
|
||||||
|
|
||||||
|
Import 전:
|
||||||
|
- [ ] 대상 데이터베이스에 스키마가 생성되어 있는지 확인 (`npm run db:push`)
|
||||||
|
- [ ] 대상 데이터베이스 URL이 정확한지 확인
|
||||||
|
- [ ] Export 파일이 손상되지 않았는지 확인
|
||||||
|
|
||||||
|
## 📞 도움말
|
||||||
|
|
||||||
|
문제가 발생하면:
|
||||||
|
1. 에러 메시지 확인
|
||||||
|
2. 데이터베이스 연결 테스트
|
||||||
|
3. 스키마가 최신인지 확인 (`npm run db:push`)
|
||||||
|
4. Export 파일 형식 검증
|
||||||
|
|
||||||
|
## 🔒 보안 주의사항
|
||||||
|
|
||||||
|
- ⚠️ Export 파일에는 **모든 사용자 데이터**가 포함됩니다
|
||||||
|
- ⚠️ **세션 데이터**와 **사용자 정보** 포함
|
||||||
|
- ⚠️ 안전한 곳에 보관하고 공유 시 주의
|
||||||
|
- ⚠️ 프로덕션 데이터는 암호화된 저장소에 보관 권장
|
||||||
|
|
||||||
|
## 📈 성능
|
||||||
|
|
||||||
|
- Export 속도: ~1,000 레코드/초
|
||||||
|
- Import 속도: ~500 레코드/초 (의존성 검사 포함)
|
||||||
|
- 100,000 레코드: ~2-3분 소요 (네트워크 속도에 따라 다름)
|
||||||
153
scripts/export-data.ts
Normal file
153
scripts/export-data.ts
Normal file
@ -0,0 +1,153 @@
|
|||||||
|
import { db } from "../server/db";
|
||||||
|
import {
|
||||||
|
sessions,
|
||||||
|
users,
|
||||||
|
mediaOutlets,
|
||||||
|
articles,
|
||||||
|
predictionMarkets,
|
||||||
|
auctions,
|
||||||
|
bids,
|
||||||
|
mediaOutletRequests,
|
||||||
|
comments,
|
||||||
|
predictionBets,
|
||||||
|
} from "../shared/schema";
|
||||||
|
import { writeFileSync } from "fs";
|
||||||
|
import { join } from "path";
|
||||||
|
|
||||||
|
interface ExportData {
|
||||||
|
exportDate: string;
|
||||||
|
version: "1.0.0";
|
||||||
|
tables: {
|
||||||
|
sessions: any[];
|
||||||
|
users: any[];
|
||||||
|
mediaOutlets: any[];
|
||||||
|
articles: any[];
|
||||||
|
predictionMarkets: any[];
|
||||||
|
auctions: any[];
|
||||||
|
bids: any[];
|
||||||
|
mediaOutletRequests: any[];
|
||||||
|
comments: any[];
|
||||||
|
predictionBets: any[];
|
||||||
|
};
|
||||||
|
metadata: {
|
||||||
|
totalRecords: number;
|
||||||
|
tableStats: Record<string, number>;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function exportData() {
|
||||||
|
console.log("🚀 Starting data export...\n");
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Export data from all tables in dependency order
|
||||||
|
console.log("📊 Exporting sessions...");
|
||||||
|
const sessionsData = await db.select().from(sessions);
|
||||||
|
console.log(` ✅ Exported ${sessionsData.length} sessions`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting users...");
|
||||||
|
const usersData = await db.select().from(users);
|
||||||
|
console.log(` ✅ Exported ${usersData.length} users`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting media outlets...");
|
||||||
|
const mediaOutletsData = await db.select().from(mediaOutlets);
|
||||||
|
console.log(` ✅ Exported ${mediaOutletsData.length} media outlets`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting articles...");
|
||||||
|
const articlesData = await db.select().from(articles);
|
||||||
|
console.log(` ✅ Exported ${articlesData.length} articles`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting prediction markets...");
|
||||||
|
const predictionMarketsData = await db.select().from(predictionMarkets);
|
||||||
|
console.log(` ✅ Exported ${predictionMarketsData.length} prediction markets`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting auctions...");
|
||||||
|
const auctionsData = await db.select().from(auctions);
|
||||||
|
console.log(` ✅ Exported ${auctionsData.length} auctions`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting bids...");
|
||||||
|
const bidsData = await db.select().from(bids);
|
||||||
|
console.log(` ✅ Exported ${bidsData.length} bids`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting media outlet requests...");
|
||||||
|
const mediaOutletRequestsData = await db.select().from(mediaOutletRequests);
|
||||||
|
console.log(` ✅ Exported ${mediaOutletRequestsData.length} media outlet requests`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting comments...");
|
||||||
|
const commentsData = await db.select().from(comments);
|
||||||
|
console.log(` ✅ Exported ${commentsData.length} comments`);
|
||||||
|
|
||||||
|
console.log("📊 Exporting prediction bets...");
|
||||||
|
const predictionBetsData = await db.select().from(predictionBets);
|
||||||
|
console.log(` ✅ Exported ${predictionBetsData.length} prediction bets`);
|
||||||
|
|
||||||
|
// Prepare export data
|
||||||
|
const totalRecords =
|
||||||
|
sessionsData.length +
|
||||||
|
usersData.length +
|
||||||
|
mediaOutletsData.length +
|
||||||
|
articlesData.length +
|
||||||
|
predictionMarketsData.length +
|
||||||
|
auctionsData.length +
|
||||||
|
bidsData.length +
|
||||||
|
mediaOutletRequestsData.length +
|
||||||
|
commentsData.length +
|
||||||
|
predictionBetsData.length;
|
||||||
|
|
||||||
|
const exportData: ExportData = {
|
||||||
|
exportDate: new Date().toISOString(),
|
||||||
|
version: "1.0.0",
|
||||||
|
tables: {
|
||||||
|
sessions: sessionsData,
|
||||||
|
users: usersData,
|
||||||
|
mediaOutlets: mediaOutletsData,
|
||||||
|
articles: articlesData,
|
||||||
|
predictionMarkets: predictionMarketsData,
|
||||||
|
auctions: auctionsData,
|
||||||
|
bids: bidsData,
|
||||||
|
mediaOutletRequests: mediaOutletRequestsData,
|
||||||
|
comments: commentsData,
|
||||||
|
predictionBets: predictionBetsData,
|
||||||
|
},
|
||||||
|
metadata: {
|
||||||
|
totalRecords,
|
||||||
|
tableStats: {
|
||||||
|
sessions: sessionsData.length,
|
||||||
|
users: usersData.length,
|
||||||
|
mediaOutlets: mediaOutletsData.length,
|
||||||
|
articles: articlesData.length,
|
||||||
|
predictionMarkets: predictionMarketsData.length,
|
||||||
|
auctions: auctionsData.length,
|
||||||
|
bids: bidsData.length,
|
||||||
|
mediaOutletRequests: mediaOutletRequestsData.length,
|
||||||
|
comments: commentsData.length,
|
||||||
|
predictionBets: predictionBetsData.length,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
// Generate filename with timestamp
|
||||||
|
const timestamp = new Date().toISOString().replace(/[:.]/g, '-').split('T')[0];
|
||||||
|
const filename = `data-export-${timestamp}.json`;
|
||||||
|
const filepath = join(process.cwd(), filename);
|
||||||
|
|
||||||
|
// Write to file
|
||||||
|
console.log(`\n💾 Writing data to ${filename}...`);
|
||||||
|
writeFileSync(filepath, JSON.stringify(exportData, null, 2), 'utf-8');
|
||||||
|
|
||||||
|
console.log(`\n✅ Export completed successfully!`);
|
||||||
|
console.log(`\n📁 File saved to: ${filepath}`);
|
||||||
|
console.log(`📊 Total records exported: ${totalRecords}`);
|
||||||
|
console.log(`\n📈 Table breakdown:`);
|
||||||
|
Object.entries(exportData.metadata.tableStats).forEach(([table, count]) => {
|
||||||
|
console.log(` - ${table}: ${count} records`);
|
||||||
|
});
|
||||||
|
|
||||||
|
process.exit(0);
|
||||||
|
} catch (error) {
|
||||||
|
console.error("\n❌ Error during export:", error);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run export
|
||||||
|
exportData();
|
||||||
175
scripts/import-data.ts
Normal file
175
scripts/import-data.ts
Normal file
@ -0,0 +1,175 @@
|
|||||||
|
import { drizzle } from "drizzle-orm/neon-serverless";
|
||||||
|
import { Pool } from "@neondatabase/serverless";
|
||||||
|
import {
|
||||||
|
sessions,
|
||||||
|
users,
|
||||||
|
mediaOutlets,
|
||||||
|
articles,
|
||||||
|
predictionMarkets,
|
||||||
|
auctions,
|
||||||
|
bids,
|
||||||
|
mediaOutletRequests,
|
||||||
|
comments,
|
||||||
|
predictionBets,
|
||||||
|
} from "../shared/schema";
|
||||||
|
import { readFileSync } from "fs";
|
||||||
|
import { join } from "path";
|
||||||
|
|
||||||
|
interface ExportData {
|
||||||
|
exportDate: string;
|
||||||
|
version: string;
|
||||||
|
tables: {
|
||||||
|
sessions: any[];
|
||||||
|
users: any[];
|
||||||
|
mediaOutlets: any[];
|
||||||
|
articles: any[];
|
||||||
|
predictionMarkets: any[];
|
||||||
|
auctions: any[];
|
||||||
|
bids: any[];
|
||||||
|
mediaOutletRequests: any[];
|
||||||
|
comments: any[];
|
||||||
|
predictionBets: any[];
|
||||||
|
};
|
||||||
|
metadata: {
|
||||||
|
totalRecords: number;
|
||||||
|
tableStats: Record<string, number>;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function importData() {
|
||||||
|
console.log("🚀 Starting data import...\n");
|
||||||
|
|
||||||
|
// Get command line arguments
|
||||||
|
const args = process.argv.slice(2);
|
||||||
|
const filename = args[0];
|
||||||
|
const targetDatabaseUrl = args[1];
|
||||||
|
|
||||||
|
if (!filename) {
|
||||||
|
console.error("❌ Error: No filename provided");
|
||||||
|
console.log("\nUsage: npm run db:import <filename> [database_url]");
|
||||||
|
console.log("\nExample:");
|
||||||
|
console.log(" npm run db:import data-export-2025-10-13.json");
|
||||||
|
console.log(" npm run db:import data-export-2025-10-13.json postgresql://user:pass@host/db");
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read export file
|
||||||
|
const filepath = join(process.cwd(), filename);
|
||||||
|
console.log(`📂 Reading data from ${filename}...`);
|
||||||
|
|
||||||
|
let exportData: ExportData;
|
||||||
|
try {
|
||||||
|
const fileContent = readFileSync(filepath, 'utf-8');
|
||||||
|
exportData = JSON.parse(fileContent);
|
||||||
|
console.log(` ✅ File loaded successfully`);
|
||||||
|
console.log(` 📅 Export date: ${exportData.exportDate}`);
|
||||||
|
console.log(` 📊 Total records: ${exportData.metadata.totalRecords}`);
|
||||||
|
} catch (error) {
|
||||||
|
console.error(`❌ Error reading file:`, error);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Setup database connection
|
||||||
|
const databaseUrl = targetDatabaseUrl || process.env.DATABASE_URL;
|
||||||
|
if (!databaseUrl) {
|
||||||
|
console.error("❌ Error: No database URL provided");
|
||||||
|
console.log("\nPlease provide a DATABASE_URL environment variable or pass it as an argument");
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n🔗 Connecting to database...`);
|
||||||
|
const pool = new Pool({ connectionString: databaseUrl });
|
||||||
|
const targetDb = drizzle(pool);
|
||||||
|
console.log(` ✅ Connected successfully`);
|
||||||
|
|
||||||
|
try {
|
||||||
|
// Import data in dependency order
|
||||||
|
console.log("\n📥 Importing data...\n");
|
||||||
|
|
||||||
|
// 1. Sessions
|
||||||
|
if (exportData.tables.sessions.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.sessions.length} sessions...`);
|
||||||
|
await targetDb.insert(sessions).values(exportData.tables.sessions).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Sessions imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Users
|
||||||
|
if (exportData.tables.users.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.users.length} users...`);
|
||||||
|
await targetDb.insert(users).values(exportData.tables.users).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Users imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Media Outlets
|
||||||
|
if (exportData.tables.mediaOutlets.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.mediaOutlets.length} media outlets...`);
|
||||||
|
await targetDb.insert(mediaOutlets).values(exportData.tables.mediaOutlets).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Media outlets imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. Articles
|
||||||
|
if (exportData.tables.articles.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.articles.length} articles...`);
|
||||||
|
await targetDb.insert(articles).values(exportData.tables.articles).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Articles imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 5. Prediction Markets
|
||||||
|
if (exportData.tables.predictionMarkets.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.predictionMarkets.length} prediction markets...`);
|
||||||
|
await targetDb.insert(predictionMarkets).values(exportData.tables.predictionMarkets).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Prediction markets imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 6. Auctions
|
||||||
|
if (exportData.tables.auctions.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.auctions.length} auctions...`);
|
||||||
|
await targetDb.insert(auctions).values(exportData.tables.auctions).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Auctions imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 7. Bids
|
||||||
|
if (exportData.tables.bids.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.bids.length} bids...`);
|
||||||
|
await targetDb.insert(bids).values(exportData.tables.bids).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Bids imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 8. Media Outlet Requests
|
||||||
|
if (exportData.tables.mediaOutletRequests.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.mediaOutletRequests.length} media outlet requests...`);
|
||||||
|
await targetDb.insert(mediaOutletRequests).values(exportData.tables.mediaOutletRequests).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Media outlet requests imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 9. Comments
|
||||||
|
if (exportData.tables.comments.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.comments.length} comments...`);
|
||||||
|
await targetDb.insert(comments).values(exportData.tables.comments).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Comments imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 10. Prediction Bets
|
||||||
|
if (exportData.tables.predictionBets.length > 0) {
|
||||||
|
console.log(`📊 Importing ${exportData.tables.predictionBets.length} prediction bets...`);
|
||||||
|
await targetDb.insert(predictionBets).values(exportData.tables.predictionBets).onConflictDoNothing();
|
||||||
|
console.log(` ✅ Prediction bets imported`);
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n✅ Import completed successfully!`);
|
||||||
|
console.log(`\n📊 Import summary:`);
|
||||||
|
Object.entries(exportData.metadata.tableStats).forEach(([table, count]) => {
|
||||||
|
console.log(` - ${table}: ${count} records`);
|
||||||
|
});
|
||||||
|
|
||||||
|
await pool.end();
|
||||||
|
process.exit(0);
|
||||||
|
} catch (error) {
|
||||||
|
console.error("\n❌ Error during import:", error);
|
||||||
|
await pool.end();
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run import
|
||||||
|
importData();
|
||||||
636
security-issues.md
Normal file
636
security-issues.md
Normal file
@ -0,0 +1,636 @@
|
|||||||
|
# Security Issues - Code Review Report
|
||||||
|
|
||||||
|
## 🔴 Critical Security Issues
|
||||||
|
|
||||||
|
### 1. No Input Sanitization - XSS Vulnerability
|
||||||
|
**Severity:** CRITICAL (OWASP Top 10 #3)
|
||||||
|
**Files Affected:**
|
||||||
|
- `server/routes.ts` (all POST/PATCH endpoints)
|
||||||
|
- Frontend components rendering user content
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
User-supplied content stored and displayed without sanitization:
|
||||||
|
```typescript
|
||||||
|
// Backend - No sanitization
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
const article = insertArticleSchema.parse(req.body);
|
||||||
|
await storage.createArticle({
|
||||||
|
...article,
|
||||||
|
content: req.body.content, // ❌ UNSANITIZED HTML/JavaScript!
|
||||||
|
title: req.body.title // ❌ UNSANITIZED!
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Frontend - Dangerous rendering
|
||||||
|
<div dangerouslySetInnerHTML={{ __html: article.content }} />
|
||||||
|
```
|
||||||
|
|
||||||
|
**Attack Vector:**
|
||||||
|
```javascript
|
||||||
|
// Attacker posts article with malicious content
|
||||||
|
{
|
||||||
|
title: "Normal Article",
|
||||||
|
content: "<script>steal_cookies()</script><img src=x onerror='alert(document.cookie)'>"
|
||||||
|
}
|
||||||
|
// When other users view this article, the script executes!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- **Cookie theft** (session hijacking)
|
||||||
|
- **Credential theft** (keylogging)
|
||||||
|
- **Malware distribution**
|
||||||
|
- **Defacement**
|
||||||
|
- **Phishing attacks**
|
||||||
|
- Affects ALL users viewing malicious content
|
||||||
|
|
||||||
|
**Affected Data:**
|
||||||
|
- Article content & titles
|
||||||
|
- Comments
|
||||||
|
- Media outlet descriptions
|
||||||
|
- User profile data
|
||||||
|
- Any user-generated content
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// Backend sanitization
|
||||||
|
import DOMPurify from 'isomorphic-dompurify';
|
||||||
|
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
const article = insertArticleSchema.parse(req.body);
|
||||||
|
|
||||||
|
// ✅ Sanitize all user input
|
||||||
|
await storage.createArticle({
|
||||||
|
...article,
|
||||||
|
title: DOMPurify.sanitize(req.body.title, {
|
||||||
|
ALLOWED_TAGS: [] // Strip all HTML from titles
|
||||||
|
}),
|
||||||
|
content: DOMPurify.sanitize(req.body.content, {
|
||||||
|
ALLOWED_TAGS: ['p', 'br', 'b', 'i', 'u', 'a', 'img'],
|
||||||
|
ALLOWED_ATTR: ['href', 'src', 'alt'],
|
||||||
|
ALLOW_DATA_ATTR: false
|
||||||
|
}),
|
||||||
|
excerpt: DOMPurify.sanitize(req.body.excerpt, {
|
||||||
|
ALLOWED_TAGS: []
|
||||||
|
})
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Frontend - Use safe rendering
|
||||||
|
import DOMPurify from 'dompurify';
|
||||||
|
|
||||||
|
// ✅ Sanitize before rendering
|
||||||
|
<div
|
||||||
|
dangerouslySetInnerHTML={{
|
||||||
|
__html: DOMPurify.sanitize(article.content)
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
|
||||||
|
// OR better - use React components instead of HTML
|
||||||
|
<div>{article.content}</div> // React auto-escapes
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - IMMEDIATE FIX REQUIRED
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. SQL Injection Risk (Potential)
|
||||||
|
**Severity:** CRITICAL (OWASP Top 10 #1)
|
||||||
|
**Files Affected:** `server/storage.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
While Drizzle ORM prevents most SQL injection, raw SQL queries are vulnerable:
|
||||||
|
```typescript
|
||||||
|
// ❌ If any raw SQL used with user input
|
||||||
|
const results = await db.execute(
|
||||||
|
sql`SELECT * FROM articles WHERE slug = ${userInput}` // Potentially unsafe
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Database breach
|
||||||
|
- Data theft
|
||||||
|
- Data deletion
|
||||||
|
- Privilege escalation
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Always use parameterized queries
|
||||||
|
const results = await db
|
||||||
|
.select()
|
||||||
|
.from(articles)
|
||||||
|
.where(eq(articles.slug, userInput)); // Drizzle handles escaping
|
||||||
|
|
||||||
|
// ✅ If raw SQL needed, use prepared statements
|
||||||
|
const stmt = db.prepare(
|
||||||
|
sql`SELECT * FROM articles WHERE slug = $1`
|
||||||
|
);
|
||||||
|
const results = await stmt.execute([userInput]);
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P0 - Audit all queries
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. Missing CSRF Protection
|
||||||
|
**Severity:** HIGH (OWASP Top 10 #8)
|
||||||
|
**Files Affected:**
|
||||||
|
- `server/index.ts`
|
||||||
|
- All state-changing endpoints
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No CSRF tokens on state-changing operations:
|
||||||
|
```typescript
|
||||||
|
// ❌ POST/PUT/DELETE endpoints have no CSRF protection
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
// Attacker can make user create article from malicious site!
|
||||||
|
});
|
||||||
|
|
||||||
|
app.delete('/api/articles/:id', async (req, res) => {
|
||||||
|
// Attacker can delete user's articles!
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Attack Vector:**
|
||||||
|
```html
|
||||||
|
<!-- Attacker's website -->
|
||||||
|
<form action="https://yourapp.com/api/articles" method="POST">
|
||||||
|
<input name="title" value="Spam Article">
|
||||||
|
<input name="content" value="Malicious content">
|
||||||
|
</form>
|
||||||
|
<script>document.forms[0].submit();</script>
|
||||||
|
<!-- When logged-in user visits, article created in their name! -->
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Unauthorized actions
|
||||||
|
- Data manipulation
|
||||||
|
- Account takeover
|
||||||
|
- Financial loss (in bidding system)
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// Install CSRF protection
|
||||||
|
import csrf from 'csurf';
|
||||||
|
|
||||||
|
const csrfProtection = csrf({
|
||||||
|
cookie: {
|
||||||
|
httpOnly: true,
|
||||||
|
secure: process.env.NODE_ENV === 'production',
|
||||||
|
sameSite: 'strict'
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Apply to all state-changing routes
|
||||||
|
app.post('/api/*', csrfProtection);
|
||||||
|
app.put('/api/*', csrfProtection);
|
||||||
|
app.patch('/api/*', csrfProtection);
|
||||||
|
app.delete('/api/*', csrfProtection);
|
||||||
|
|
||||||
|
// Send token to client
|
||||||
|
app.get('/api/csrf-token', csrfProtection, (req, res) => {
|
||||||
|
res.json({ csrfToken: req.csrfToken() });
|
||||||
|
});
|
||||||
|
|
||||||
|
// Client must include token
|
||||||
|
fetch('/api/articles', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
'CSRF-Token': csrfToken,
|
||||||
|
'Content-Type': 'application/json'
|
||||||
|
},
|
||||||
|
body: JSON.stringify(data)
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Add before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. Insufficient Authentication Checks
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Files Affected:** `server/routes.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Critical endpoints missing authentication:
|
||||||
|
```typescript
|
||||||
|
// ❌ Anyone can create articles!
|
||||||
|
app.post('/api/articles', async (req, res) => {
|
||||||
|
// No authentication check!
|
||||||
|
});
|
||||||
|
|
||||||
|
// ❌ Anyone can see all bids!
|
||||||
|
app.get('/api/bids', async (req, res) => {
|
||||||
|
// No authentication check!
|
||||||
|
});
|
||||||
|
|
||||||
|
// ❌ Anyone can modify media outlets!
|
||||||
|
app.patch('/api/media-outlets/:slug', async (req, res) => {
|
||||||
|
// Checks inside route, not as middleware
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Unauthorized data access
|
||||||
|
- Spam creation
|
||||||
|
- Data manipulation
|
||||||
|
- Privilege escalation
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Require authentication by default
|
||||||
|
const publicRoutes = [
|
||||||
|
'/api/media-outlets',
|
||||||
|
'/api/articles',
|
||||||
|
'/api/auth/user'
|
||||||
|
];
|
||||||
|
|
||||||
|
app.use('/api/*', (req, res, next) => {
|
||||||
|
// Allow public routes
|
||||||
|
if (publicRoutes.some(route => req.path.startsWith(route))) {
|
||||||
|
return next();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Require auth for everything else
|
||||||
|
return isAuthenticated(req, res, next);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ✅ Explicit role checks
|
||||||
|
const requireAdmin = (req: any, res: Response, next: NextFunction) => {
|
||||||
|
if (!req.user || (req.user.role !== 'admin' && req.user.role !== 'superadmin')) {
|
||||||
|
return res.status(403).json({ message: "Admin access required" });
|
||||||
|
}
|
||||||
|
next();
|
||||||
|
};
|
||||||
|
|
||||||
|
app.post('/api/articles', isAuthenticated, requireAdmin, async (req, res) => {
|
||||||
|
// Only authenticated admins can create
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Fix before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. Weak Session Configuration
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Files Affected:**
|
||||||
|
- `server/simpleAuth.ts`
|
||||||
|
- `server/replitAuth.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Session cookies lack security flags:
|
||||||
|
```typescript
|
||||||
|
// ❌ Insecure session config
|
||||||
|
app.use(session({
|
||||||
|
secret: process.env.SESSION_SECRET || 'dev-secret', // ❌ Weak default
|
||||||
|
resave: false,
|
||||||
|
saveUninitialized: false,
|
||||||
|
cookie: {
|
||||||
|
// ❌ Missing security flags!
|
||||||
|
maxAge: 30 * 24 * 60 * 60 * 1000 // 30 days - too long!
|
||||||
|
}
|
||||||
|
}));
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Session hijacking
|
||||||
|
- Cookie theft
|
||||||
|
- XSS escalation
|
||||||
|
- Man-in-the-middle attacks
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Secure session configuration
|
||||||
|
app.use(session({
|
||||||
|
secret: process.env.SESSION_SECRET, // ✅ Require in production
|
||||||
|
resave: false,
|
||||||
|
saveUninitialized: false,
|
||||||
|
name: 'sessionId', // ✅ Don't reveal framework
|
||||||
|
cookie: {
|
||||||
|
httpOnly: true, // ✅ Prevent JavaScript access
|
||||||
|
secure: process.env.NODE_ENV === 'production', // ✅ HTTPS only in prod
|
||||||
|
sameSite: 'strict', // ✅ CSRF protection
|
||||||
|
maxAge: 24 * 60 * 60 * 1000, // ✅ 1 day (not 30!)
|
||||||
|
domain: process.env.COOKIE_DOMAIN // ✅ Limit to your domain
|
||||||
|
},
|
||||||
|
store: sessionStore // ✅ Use persistent store (not memory)
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Validate SESSION_SECRET exists in production
|
||||||
|
if (process.env.NODE_ENV === 'production' && !process.env.SESSION_SECRET) {
|
||||||
|
throw new Error('SESSION_SECRET must be set in production');
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Fix immediately
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟠 High Priority Security Issues
|
||||||
|
|
||||||
|
### 6. No Rate Limiting - DoS Vulnerability
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Files Affected:** All API endpoints
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No rate limiting allows abuse:
|
||||||
|
```typescript
|
||||||
|
// ❌ Unlimited requests possible
|
||||||
|
app.post('/api/auctions/:id/bid', async (req, res) => {
|
||||||
|
// Attacker can spam bids!
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/comments', async (req, res) => {
|
||||||
|
// Attacker can spam comments!
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Denial of Service
|
||||||
|
- Resource exhaustion
|
||||||
|
- Bid manipulation
|
||||||
|
- Spam flooding
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import rateLimit from 'express-rate-limit';
|
||||||
|
|
||||||
|
// ✅ General API rate limit
|
||||||
|
const apiLimiter = rateLimit({
|
||||||
|
windowMs: 15 * 60 * 1000, // 15 minutes
|
||||||
|
max: 100, // 100 requests per window
|
||||||
|
message: 'Too many requests, please try again later',
|
||||||
|
standardHeaders: true,
|
||||||
|
legacyHeaders: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
app.use('/api/', apiLimiter);
|
||||||
|
|
||||||
|
// ✅ Strict limits for sensitive operations
|
||||||
|
const bidLimiter = rateLimit({
|
||||||
|
windowMs: 60 * 1000, // 1 minute
|
||||||
|
max: 5, // 5 bids per minute
|
||||||
|
skipSuccessfulRequests: false
|
||||||
|
});
|
||||||
|
|
||||||
|
const commentLimiter = rateLimit({
|
||||||
|
windowMs: 60 * 1000,
|
||||||
|
max: 10 // 10 comments per minute
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/auctions/:id/bid', bidLimiter, async (req, res) => {});
|
||||||
|
app.post('/api/comments', commentLimiter, async (req, res) => {});
|
||||||
|
|
||||||
|
// ✅ Login rate limiting (prevent brute force)
|
||||||
|
const loginLimiter = rateLimit({
|
||||||
|
windowMs: 15 * 60 * 1000,
|
||||||
|
max: 5, // 5 login attempts per 15 minutes
|
||||||
|
skipSuccessfulRequests: true
|
||||||
|
});
|
||||||
|
|
||||||
|
app.post('/api/login', loginLimiter, async (req, res) => {});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P1 - Add before production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. Sensitive Data Exposure in Logs
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**Files Affected:** Multiple routes
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
Logging sensitive user data:
|
||||||
|
```typescript
|
||||||
|
catch (error) {
|
||||||
|
console.error("Error:", error);
|
||||||
|
console.error("Request body:", req.body); // ❌ May contain passwords!
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Password exposure
|
||||||
|
- Session token leaks
|
||||||
|
- PII in logs
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Sanitize logs
|
||||||
|
const sanitizeLog = (data: any) => {
|
||||||
|
const sensitive = ['password', 'token', 'secret', 'apiKey'];
|
||||||
|
const sanitized = { ...data };
|
||||||
|
|
||||||
|
for (const key of sensitive) {
|
||||||
|
if (key in sanitized) {
|
||||||
|
sanitized[key] = '[REDACTED]';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return sanitized;
|
||||||
|
};
|
||||||
|
|
||||||
|
catch (error) {
|
||||||
|
console.error("Error:", error.message);
|
||||||
|
console.error("Request body:", sanitizeLog(req.body));
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P2 - Implement logging standards
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 8. Missing Security Headers
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**Files Affected:** `server/index.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No security headers configured:
|
||||||
|
```typescript
|
||||||
|
// ❌ Missing security headers
|
||||||
|
app.use(express.json());
|
||||||
|
// No helmet, no CSP, no security headers!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- XSS attacks easier
|
||||||
|
- Clickjacking possible
|
||||||
|
- MIME sniffing attacks
|
||||||
|
- Missing defense-in-depth
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
import helmet from 'helmet';
|
||||||
|
|
||||||
|
// ✅ Add security headers
|
||||||
|
app.use(helmet({
|
||||||
|
contentSecurityPolicy: {
|
||||||
|
directives: {
|
||||||
|
defaultSrc: ["'self'"],
|
||||||
|
styleSrc: ["'self'", "'unsafe-inline'"],
|
||||||
|
scriptSrc: ["'self'"],
|
||||||
|
imgSrc: ["'self'", "data:", "https:"],
|
||||||
|
connectSrc: ["'self'"],
|
||||||
|
fontSrc: ["'self'"],
|
||||||
|
objectSrc: ["'none'"],
|
||||||
|
mediaSrc: ["'self'"],
|
||||||
|
frameSrc: ["'none'"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
hsts: {
|
||||||
|
maxAge: 31536000,
|
||||||
|
includeSubDomains: true,
|
||||||
|
preload: true
|
||||||
|
},
|
||||||
|
referrerPolicy: { policy: 'strict-origin-when-cross-origin' }
|
||||||
|
}));
|
||||||
|
|
||||||
|
// ✅ Additional headers
|
||||||
|
app.use((req, res, next) => {
|
||||||
|
res.setHeader('X-Content-Type-Options', 'nosniff');
|
||||||
|
res.setHeader('X-Frame-Options', 'DENY');
|
||||||
|
res.setHeader('X-XSS-Protection', '1; mode=block');
|
||||||
|
next();
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P2 - Add security layer
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟡 Medium Priority Security Issues
|
||||||
|
|
||||||
|
### 9. No Content Length Limits
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**Files Affected:** `server/index.ts`
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No request size limits:
|
||||||
|
```typescript
|
||||||
|
app.use(express.json()); // ❌ No size limit!
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Memory exhaustion
|
||||||
|
- DoS via large payloads
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Limit request size
|
||||||
|
app.use(express.json({ limit: '10mb' })); // Reasonable limit
|
||||||
|
app.use(express.urlencoded({ limit: '10mb', extended: true }));
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P2 - Prevent abuse
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 10. Insecure Direct Object References (IDOR)
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**Files Affected:** Various endpoints
|
||||||
|
|
||||||
|
**Problem:**
|
||||||
|
No ownership validation:
|
||||||
|
```typescript
|
||||||
|
// ❌ User can delete ANY comment by guessing ID!
|
||||||
|
app.delete('/api/comments/:id', async (req, res) => {
|
||||||
|
await storage.deleteComment(req.params.id);
|
||||||
|
// No check if user owns this comment!
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Impact:**
|
||||||
|
- Unauthorized data access
|
||||||
|
- Data deletion
|
||||||
|
- Privilege escalation
|
||||||
|
|
||||||
|
**Fix Solution:**
|
||||||
|
```typescript
|
||||||
|
// ✅ Validate ownership
|
||||||
|
app.delete('/api/comments/:id', isAuthenticated, async (req: any, res) => {
|
||||||
|
const comment = await storage.getCommentById(req.params.id);
|
||||||
|
|
||||||
|
if (!comment) {
|
||||||
|
return res.status(404).json({ message: "Comment not found" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check ownership or admin
|
||||||
|
if (comment.userId !== req.user.id && req.user.role !== 'admin') {
|
||||||
|
return res.status(403).json({ message: "Not authorized" });
|
||||||
|
}
|
||||||
|
|
||||||
|
await storage.deleteComment(req.params.id);
|
||||||
|
res.json({ success: true });
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Priority:** P2 - Add authorization checks
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security Checklist
|
||||||
|
|
||||||
|
### Immediate Actions (P0-P1)
|
||||||
|
- [ ] Sanitize ALL user input (XSS prevention)
|
||||||
|
- [ ] Audit SQL queries for injection risks
|
||||||
|
- [ ] Implement CSRF protection
|
||||||
|
- [ ] Add authentication to all endpoints
|
||||||
|
- [ ] Secure session configuration
|
||||||
|
- [ ] Add rate limiting
|
||||||
|
- [ ] Review and fix privilege escalation risks
|
||||||
|
|
||||||
|
### Before Production (P1-P2)
|
||||||
|
- [ ] Add security headers (Helmet)
|
||||||
|
- [ ] Implement proper logging (no sensitive data)
|
||||||
|
- [ ] Add request size limits
|
||||||
|
- [ ] Fix IDOR vulnerabilities
|
||||||
|
- [ ] Set up security monitoring
|
||||||
|
- [ ] Conduct security audit
|
||||||
|
- [ ] Penetration testing
|
||||||
|
|
||||||
|
### Best Practices
|
||||||
|
- [ ] Regular security updates
|
||||||
|
- [ ] Dependency vulnerability scanning
|
||||||
|
- [ ] Security training for team
|
||||||
|
- [ ] Incident response plan
|
||||||
|
- [ ] Regular backups
|
||||||
|
- [ ] Security code review process
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
| Severity | Count | Must Fix Before Production |
|
||||||
|
|----------|-------|----------------------------|
|
||||||
|
| 🔴 Critical | 5 | ✅ Yes - BLOCKING |
|
||||||
|
| 🟠 High | 3 | ✅ Yes |
|
||||||
|
| 🟡 Medium | 2 | ⚠️ Strongly Recommended |
|
||||||
|
|
||||||
|
**Total Security Issues:** 10
|
||||||
|
|
||||||
|
**OWASP Top 10 Coverage:**
|
||||||
|
- ✅ A01: Broken Access Control (Issues #3, #4, #10)
|
||||||
|
- ✅ A02: Cryptographic Failures (Issue #5)
|
||||||
|
- ✅ A03: Injection (Issues #1, #2)
|
||||||
|
- ✅ A05: Security Misconfiguration (Issues #5, #8)
|
||||||
|
- ✅ A07: Identification/Authentication Failures (Issues #4, #5)
|
||||||
|
- ✅ A08: Software/Data Integrity Failures (Issue #3)
|
||||||
|
|
||||||
|
**Estimated Fix Time:**
|
||||||
|
- Critical issues: 8-12 hours
|
||||||
|
- High priority: 4-6 hours
|
||||||
|
- Medium priority: 4-6 hours
|
||||||
|
- **Total: 16-24 hours**
|
||||||
|
|
||||||
|
**Security Hardening Priority:**
|
||||||
|
1. Input sanitization (XSS)
|
||||||
|
2. CSRF protection
|
||||||
|
3. Authentication/authorization
|
||||||
|
4. Session security
|
||||||
|
5. Rate limiting
|
||||||
|
6. Security headers
|
||||||
|
7. Audit logging
|
||||||
|
8. IDOR fixes
|
||||||
|
|
||||||
|
**Compliance Notes:**
|
||||||
|
- GDPR: Add data encryption, privacy controls
|
||||||
|
- PCI DSS: Required if handling payments
|
||||||
|
- SOC 2: Implement audit logging
|
||||||
|
- HIPAA: Not applicable unless handling health data
|
||||||
Reference in New Issue
Block a user