done
This commit is contained in:
305
README.md
Normal file
305
README.md
Normal file
@@ -0,0 +1,305 @@
|
||||
# Secure Video & File Sharing Management System
|
||||
|
||||
A production-grade, enterprise-ready full-stack web application designed for secure video streaming, file management, granular administrator-controlled sharing, and multi-tenant access control.
|
||||
|
||||
Engineered from the ground up to run seamlessly and natively across **both Linux (Ubuntu, Debian, CentOS, RHEL, Arch) and Windows (Windows 10/11, Windows Server)**, as well as containerized in Docker.
|
||||
|
||||
---
|
||||
|
||||
## 🌟 Key Features
|
||||
|
||||
1. **Dual-OS Filesystem Architecture (Linux & Windows)**:
|
||||
- Cross-platform storage engine (`StoragePathUtils` and `LocalStorageService`).
|
||||
- Canonical POSIX database keys (`users/{userId}/files/{fileId}.{ext}`) mapped cleanly to native host OS paths (Windows drive letters & backslashes or Linux root POSIX paths).
|
||||
- Strict directory traversal protection (`..` prevention) utilizing path-relative isolation.
|
||||
- OS-native permission enforcement (`0o755` directory mode on Linux).
|
||||
- FFmpeg auto-detection across standard Linux paths (`/usr/bin/ffmpeg`) and Windows paths (`C:\ffmpeg\bin`, WinGet, Chocolatey).
|
||||
|
||||
2. **Adaptive Video Processing & Streaming Engine**:
|
||||
- Automated FFprobe metadata extraction (resolution, duration, codecs).
|
||||
- Video thumbnail frame extraction at keyframe intervals.
|
||||
- H.264 / AAC HLS (HTTP Live Streaming) adaptive bitrate segmentation with `.m3u8` master playlists and `.ts` chunk streaming.
|
||||
- Custom in-browser HLS.js player with resolution switching, playback rate controls, scrubbing, and frame stepping.
|
||||
- Resilient background queue worker (BullMQ + Redis with fallback to in-process execution when Redis is offline).
|
||||
|
||||
3. **Multi-Tenant Security & Granular Access Control**:
|
||||
- Private user namespaces: User B cannot access User A's private files (complete IDOR protection).
|
||||
- User-wise file explorer for administrators.
|
||||
- Granular permission matrix:
|
||||
- `canViewMetadata`
|
||||
- `canPreview`
|
||||
- `canPlay` (Stream video) — **strictly does NOT grant download**
|
||||
- `canDownload` (Decrypt / stream original raw binary file)
|
||||
- `canListFolder`
|
||||
- Real-time permission revocation: immediate server-side guard enforcement.
|
||||
|
||||
4. **Storage Quota & Upload Pipeline**:
|
||||
- Atomic quota reservation: rejects uploads before receiving bytes if quota exceeded.
|
||||
- Upload session management with chunked progress tracking and instant direct uploads.
|
||||
- Magic byte validation and MIME-type inspection (preventing file spoofing).
|
||||
- Soft-delete lifecycle with immediate quota reclamation.
|
||||
|
||||
5. **Compliance & Audit Logging**:
|
||||
- Immutable chronological audit trail recording actions, actors, targets, IP addresses, user agents, and metadata payloads.
|
||||
|
||||
---
|
||||
|
||||
## ⚡ 1 Single Command to Run Everything (Windows & Linux)
|
||||
|
||||
Whether you are on **Windows** or **Linux**, you can run the entire system with just **ONE** command:
|
||||
|
||||
### 🚀 Production Mode (Backend + Frontend together):
|
||||
```bash
|
||||
npm start
|
||||
```
|
||||
|
||||
### 🛠️ Development Mode (With Hot-Reload & Watch):
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
### 🐳 Complete Docker Stack (MySQL, Redis, MinIO, Backend, Frontend):
|
||||
```bash
|
||||
npm run docker
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Technology Stack
|
||||
|
||||
| Layer | Technologies |
|
||||
|---|---|
|
||||
| **Frontend** | Next.js 15 (App Router), React 19, TypeScript, Tailwind CSS, Lucide React, HLS.js |
|
||||
| **Backend** | NestJS 11, TypeScript, Express, Helmet, CORS, Argon2id, Passport-JWT |
|
||||
| **Database & ORM** | MySQL 8.0 / MariaDB, Prisma ORM 6 |
|
||||
| **Video Engine** | FFmpeg, FFprobe, fluent-ffmpeg, HLS Transcoder |
|
||||
| **Task Queue** | BullMQ, Redis 7 (with local resilient fallback runner) |
|
||||
| **Storage Abstraction**| Local Filesystem Driver (Windows/Linux) + S3/MinIO Driver |
|
||||
| **Documentation** | Swagger / OpenAPI 3.0 (`http://localhost:4000/api/docs`) |
|
||||
|
||||
---
|
||||
|
||||
## 💻 Operating System & Filesystem Setup
|
||||
|
||||
### 🪟 Hosting on Windows (Native)
|
||||
|
||||
#### Prerequisites:
|
||||
1. **Node.js**: v20.x or v22.x LTS ([nodejs.org](https://nodejs.org))
|
||||
2. **MySQL / MariaDB**: MySQL 8.0 or XAMPP running on `127.0.0.1:3306`
|
||||
3. **FFmpeg**: Download from [gyan.dev/ffmpeg](https://www.gyan.dev/ffmpeg/builds/) or install via WinGet:
|
||||
```cmd
|
||||
winget install Gyan.FFmpeg
|
||||
```
|
||||
|
||||
#### Configuration (`.env` in `apps/backend/.env`):
|
||||
```env
|
||||
PORT=4000
|
||||
FRONTEND_URL=http://localhost:3000
|
||||
DATABASE_URL="mysql://root:@127.0.0.1:3306/fileshare_db"
|
||||
|
||||
# Windows Filesystem Path (either absolute or relative)
|
||||
STORAGE_DRIVER=local
|
||||
LOCAL_STORAGE_PATH=D:\Project\fileshareproject\storage
|
||||
# or LOCAL_STORAGE_PATH=../../storage
|
||||
```
|
||||
|
||||
#### One-Click Launch on Windows:
|
||||
Double click `start-windows.bat` or run:
|
||||
```cmd
|
||||
start-windows.bat
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🐧 Hosting on Linux (Ubuntu / Debian / CentOS / RHEL)
|
||||
|
||||
#### Prerequisites:
|
||||
```bash
|
||||
# Ubuntu / Debian
|
||||
sudo apt-get update
|
||||
sudo apt-get install -y nodejs npm ffmpeg mysql-server redis-server
|
||||
|
||||
# CentOS / RHEL
|
||||
sudo dnf install -y nodejs npm ffmpeg mysql-server redis
|
||||
```
|
||||
|
||||
#### Database Setup:
|
||||
```bash
|
||||
sudo mysql -e "CREATE DATABASE fileshare_db CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
|
||||
sudo mysql -e "CREATE USER 'fileshare'@'localhost' IDENTIFIED BY 'YourSecurePassword2026!';"
|
||||
sudo mysql -e "GRANT ALL PRIVILEGES ON fileshare_db.* TO 'fileshare'@'localhost'; FLUSH PRIVILEGES;"
|
||||
```
|
||||
|
||||
#### Configuration (`apps/backend/.env`):
|
||||
```env
|
||||
PORT=4000
|
||||
FRONTEND_URL=http://localhost:3000
|
||||
DATABASE_URL="mysql://fileshare:YourSecurePassword2026!@127.0.0.1:3306/fileshare_db"
|
||||
|
||||
# Linux Filesystem Path (Standard POSIX with 0755 permissions)
|
||||
STORAGE_DRIVER=local
|
||||
LOCAL_STORAGE_PATH=/var/lib/fileshare/storage
|
||||
# or LOCAL_STORAGE_PATH=./storage
|
||||
```
|
||||
|
||||
#### Launch on Linux:
|
||||
```bash
|
||||
chmod +x start-linux.sh
|
||||
./start-linux.sh
|
||||
```
|
||||
|
||||
#### Optional: Linux Systemd Service (`/etc/systemd/system/fileshare-backend.service`)
|
||||
```ini
|
||||
[Unit]
|
||||
Description=FileShare Backend NestJS Service
|
||||
After=network.target mysql.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=www-data
|
||||
WorkingDirectory=/var/www/fileshare/apps/backend
|
||||
ExecStart=/usr/bin/node dist/main.js
|
||||
Restart=on-failure
|
||||
Environment=NODE_ENV=production
|
||||
EnvironmentFile=/var/www/fileshare/apps/backend/.env
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🐳 Hosting with Docker & Docker Compose (Any OS)
|
||||
|
||||
Run the entire stack (MySQL 8.0, Redis 7, Backend with FFmpeg, Frontend Next.js, and MinIO S3) with a single command:
|
||||
|
||||
```bash
|
||||
# Build and launch all containers
|
||||
npm run docker:up
|
||||
# or
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
- **Frontend App**: `http://localhost:3000`
|
||||
- **Backend API & Swagger**: `http://localhost:4000/api/docs`
|
||||
- **MinIO S3 Console**: `http://localhost:9001` (User: `minioadmin` / `minioadminpassword`)
|
||||
|
||||
To view live container logs:
|
||||
```bash
|
||||
npm run docker:logs
|
||||
```
|
||||
|
||||
To stop containers:
|
||||
```bash
|
||||
npm run docker:down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 👥 Pre-Configured User Accounts
|
||||
|
||||
The database comes pre-seeded with the following active accounts:
|
||||
|
||||
| Role | Username | Email | Password | Initial Quota | Description |
|
||||
|---|---|---|---|---|---|
|
||||
| **Super Admin** | `Ecohub` | `ecohub@fileshare.local` | `Ecohub@520` | 100 GB | Full system control, quota manager, settings & audit logs |
|
||||
| **Staff Officer**| `Ecohubstaff` | `ecohubstaff@fileshare.local` | `123` | 50 GB | Staff admin, user-wise file explorer, permission grants |
|
||||
| **Normal User** | `ecouser` | `ecouser@fileshare.local` | `123` | 10 GB | Standard private vault, video streaming & sharing |
|
||||
|
||||
> *Tip: On the `/login` page, you can click on any of the 1-Click Instant Login cards to populate credentials immediately.*
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ MySQL Direct Import File (`users_import.sql`)
|
||||
|
||||
For direct database seeding without running Node scripts, a standalone MySQL import script is provided in the root directory: [`users_import.sql`](file:///d:/Project/fileshareproject/users_import.sql).
|
||||
|
||||
### How to Import into MySQL:
|
||||
|
||||
#### Option 1: Via MySQL Command Line (Windows or Linux):
|
||||
```bash
|
||||
mysql -u root fileshare_db < users_import.sql
|
||||
```
|
||||
|
||||
#### Option 2: Via phpMyAdmin or HeidiSQL:
|
||||
1. Open **phpMyAdmin** or **HeidiSQL**.
|
||||
2. Select the `fileshare_db` database.
|
||||
3. Click the **Import** tab.
|
||||
4. Choose the file [`users_import.sql`](file:///d:/Project/fileshareproject/users_import.sql) and click **Go**.
|
||||
5. All three accounts (`Ecohub`, `Ecohubstaff`, `ecouser`), their Argon2id password hashes, and their root storage folders will be created immediately!
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Automated Test Suite
|
||||
|
||||
Run the end-to-end integration and security test suites:
|
||||
|
||||
```bash
|
||||
# Access control, IDOR, and permission tests
|
||||
cd apps/backend
|
||||
npm run test:e2e
|
||||
```
|
||||
|
||||
**Tested Scenarios**:
|
||||
1. Cross-tenant isolation (User B receives `404 Not Found` when requesting User A's private file).
|
||||
2. Separation of `canPlay` and `canDownload` (User B can stream HTTP 206 chunks but receives `403 Forbidden` on download).
|
||||
3. Immediate permission revocation (granting and revoking reflects instantly on protected routes).
|
||||
4. Administrator-only routes protection (User A denied access to `/admin/*`).
|
||||
5. Video processing pipeline (FFprobe probe, thumbnail generation, HLS master playlist & TS chunks).
|
||||
|
||||
---
|
||||
|
||||
## 📁 Repository Structure
|
||||
|
||||
```
|
||||
fileshareproject/
|
||||
├── docker-compose.yml # Multi-container orchestration (MySQL, Redis, Backend, Frontend)
|
||||
├── start-windows.bat # 1-click Windows launcher
|
||||
├── start-linux.sh # 1-click Linux launcher
|
||||
├── .env.example # Comprehensive multi-environment configuration
|
||||
├── packages/
|
||||
│ └── shared/ # Shared TypeScript DTOs, enums, permission contracts
|
||||
│ ├── package.json
|
||||
│ └── src/index.ts
|
||||
├── apps/
|
||||
│ ├── backend/ # NestJS API Server
|
||||
│ │ ├── Dockerfile # Linux Debian-slim container with FFmpeg & Prisma
|
||||
│ │ ├── prisma/
|
||||
│ │ │ ├── schema.prisma # MySQL Schema (Users, Folders, FileAssets, Permissions, Logs)
|
||||
│ │ │ ├── migrations/ # Prisma migration history
|
||||
│ │ │ └── seed.ts # Initial administrative bootstrap seeder
|
||||
│ │ └── src/
|
||||
│ │ ├── modules/
|
||||
│ │ │ ├── auth/ # Argon2id password hashing, JWT access/refresh tokens
|
||||
│ │ │ ├── storage/ # Cross-platform LocalStorage & S3Storage providers
|
||||
│ │ │ ├── video/ # FFmpeg transcoder & HLS generator
|
||||
│ │ │ ├── permissions/ # Granular permission engine (file, folder, user-level)
|
||||
│ │ │ ├── files/ # Direct & chunked upload sessions, range streaming
|
||||
│ │ │ ├── folders/ # Hierarchical logical folder management
|
||||
│ │ │ ├── admin/ # User directory, storage metrics, audit trail
|
||||
│ │ │ └── audit/ # Security event logger
|
||||
│ └── frontend/ # Next.js 15 App Router Frontend
|
||||
│ ├── Dockerfile # Multi-stage production container
|
||||
│ └── src/
|
||||
│ ├── app/
|
||||
│ │ ├── dashboard/ # Storage gauges, quota bars, recent uploads
|
||||
│ │ ├── files/ # File explorer, search, category filters, upload modal
|
||||
│ │ ├── shared/ # Files shared with current user
|
||||
│ │ ├── video/[id]/ # Adaptive HLS player with custom scrubber
|
||||
│ │ └── admin/ # Users, user-files, permissions, storage, audit logs
|
||||
│ └── components/
|
||||
│ ├── video-player/ # HlsVideoPlayer component
|
||||
│ ├── uploader/ # FileUploaderModal with drag-and-drop
|
||||
│ └── file-manager/ # FileCard, FileRow, FilePreviewModal
|
||||
└── storage/ # Default local storage volume (users/{id}/files/...)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Security Summary
|
||||
|
||||
- **No Public S3/Static URLs**: All file downloads, previews, and video streams pass through authentication and authorization guards.
|
||||
- **Directory Traversal Immunity**: Storage paths use canonical keys and reject relative path components (`..`).
|
||||
- **Input Sanitization**: Original file names are cleaned of special characters and illegal OS tokens.
|
||||
- **Argon2id Hashing**: Industry-standard cryptographic password hashing.
|
||||
- **Audit Compliance**: All administrative and file modifications are tracked in database audit records.
|
||||
Reference in New Issue
Block a user