Files
project-fileshare/README.md

306 lines
12 KiB
Markdown
Raw Permalink Normal View History

2026-10-29 13:21:30 +06:00
# Secure Video & File Sharing Management System
A production-grade, enterprise-ready full-stack web application designed for secure video streaming, file management, granular administrator-controlled sharing, and multi-tenant access control.
Engineered from the ground up to run seamlessly and natively across **both Linux (Ubuntu, Debian, CentOS, RHEL, Arch) and Windows (Windows 10/11, Windows Server)**, as well as containerized in Docker.
---
## 🌟 Key Features
1. **Dual-OS Filesystem Architecture (Linux & Windows)**:
- Cross-platform storage engine (`StoragePathUtils` and `LocalStorageService`).
- Canonical POSIX database keys (`users/{userId}/files/{fileId}.{ext}`) mapped cleanly to native host OS paths (Windows drive letters & backslashes or Linux root POSIX paths).
- Strict directory traversal protection (`..` prevention) utilizing path-relative isolation.
- OS-native permission enforcement (`0o755` directory mode on Linux).
- FFmpeg auto-detection across standard Linux paths (`/usr/bin/ffmpeg`) and Windows paths (`C:\ffmpeg\bin`, WinGet, Chocolatey).
2. **Adaptive Video Processing & Streaming Engine**:
- Automated FFprobe metadata extraction (resolution, duration, codecs).
- Video thumbnail frame extraction at keyframe intervals.
- H.264 / AAC HLS (HTTP Live Streaming) adaptive bitrate segmentation with `.m3u8` master playlists and `.ts` chunk streaming.
- Custom in-browser HLS.js player with resolution switching, playback rate controls, scrubbing, and frame stepping.
- Resilient background queue worker (BullMQ + Redis with fallback to in-process execution when Redis is offline).
3. **Multi-Tenant Security & Granular Access Control**:
- Private user namespaces: User B cannot access User A's private files (complete IDOR protection).
- User-wise file explorer for administrators.
- Granular permission matrix:
- `canViewMetadata`
- `canPreview`
- `canPlay` (Stream video) — **strictly does NOT grant download**
- `canDownload` (Decrypt / stream original raw binary file)
- `canListFolder`
- Real-time permission revocation: immediate server-side guard enforcement.
4. **Storage Quota & Upload Pipeline**:
- Atomic quota reservation: rejects uploads before receiving bytes if quota exceeded.
- Upload session management with chunked progress tracking and instant direct uploads.
- Magic byte validation and MIME-type inspection (preventing file spoofing).
- Soft-delete lifecycle with immediate quota reclamation.
5. **Compliance & Audit Logging**:
- Immutable chronological audit trail recording actions, actors, targets, IP addresses, user agents, and metadata payloads.
---
## ⚡ 1 Single Command to Run Everything (Windows & Linux)
Whether you are on **Windows** or **Linux**, you can run the entire system with just **ONE** command:
### 🚀 Production Mode (Backend + Frontend together):
```bash
npm start
```
### 🛠️ Development Mode (With Hot-Reload & Watch):
```bash
npm run dev
```
### 🐳 Complete Docker Stack (MySQL, Redis, MinIO, Backend, Frontend):
```bash
npm run docker
```
---
## 🏗️ Technology Stack
| Layer | Technologies |
|---|---|
| **Frontend** | Next.js 15 (App Router), React 19, TypeScript, Tailwind CSS, Lucide React, HLS.js |
| **Backend** | NestJS 11, TypeScript, Express, Helmet, CORS, Argon2id, Passport-JWT |
| **Database & ORM** | MySQL 8.0 / MariaDB, Prisma ORM 6 |
| **Video Engine** | FFmpeg, FFprobe, fluent-ffmpeg, HLS Transcoder |
| **Task Queue** | BullMQ, Redis 7 (with local resilient fallback runner) |
| **Storage Abstraction**| Local Filesystem Driver (Windows/Linux) + S3/MinIO Driver |
| **Documentation** | Swagger / OpenAPI 3.0 (`http://localhost:4000/api/docs`) |
---
## 💻 Operating System & Filesystem Setup
### 🪟 Hosting on Windows (Native)
#### Prerequisites:
1. **Node.js**: v20.x or v22.x LTS ([nodejs.org](https://nodejs.org))
2. **MySQL / MariaDB**: MySQL 8.0 or XAMPP running on `127.0.0.1:3306`
3. **FFmpeg**: Download from [gyan.dev/ffmpeg](https://www.gyan.dev/ffmpeg/builds/) or install via WinGet:
```cmd
winget install Gyan.FFmpeg
```
#### Configuration (`.env` in `apps/backend/.env`):
```env
PORT=4000
FRONTEND_URL=http://localhost:3000
DATABASE_URL="mysql://root:@127.0.0.1:3306/fileshare_db"
# Windows Filesystem Path (either absolute or relative)
STORAGE_DRIVER=local
LOCAL_STORAGE_PATH=D:\Project\fileshareproject\storage
# or LOCAL_STORAGE_PATH=../../storage
```
#### One-Click Launch on Windows:
Double click `start-windows.bat` or run:
```cmd
start-windows.bat
```
---
### 🐧 Hosting on Linux (Ubuntu / Debian / CentOS / RHEL)
#### Prerequisites:
```bash
# Ubuntu / Debian
sudo apt-get update
sudo apt-get install -y nodejs npm ffmpeg mysql-server redis-server
# CentOS / RHEL
sudo dnf install -y nodejs npm ffmpeg mysql-server redis
```
#### Database Setup:
```bash
sudo mysql -e "CREATE DATABASE fileshare_db CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
sudo mysql -e "CREATE USER 'fileshare'@'localhost' IDENTIFIED BY 'YourSecurePassword2026!';"
sudo mysql -e "GRANT ALL PRIVILEGES ON fileshare_db.* TO 'fileshare'@'localhost'; FLUSH PRIVILEGES;"
```
#### Configuration (`apps/backend/.env`):
```env
PORT=4000
FRONTEND_URL=http://localhost:3000
DATABASE_URL="mysql://fileshare:YourSecurePassword2026!@127.0.0.1:3306/fileshare_db"
# Linux Filesystem Path (Standard POSIX with 0755 permissions)
STORAGE_DRIVER=local
LOCAL_STORAGE_PATH=/var/lib/fileshare/storage
# or LOCAL_STORAGE_PATH=./storage
```
#### Launch on Linux:
```bash
chmod +x start-linux.sh
./start-linux.sh
```
#### Optional: Linux Systemd Service (`/etc/systemd/system/fileshare-backend.service`)
```ini
[Unit]
Description=FileShare Backend NestJS Service
After=network.target mysql.service
[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/fileshare/apps/backend
ExecStart=/usr/bin/node dist/main.js
Restart=on-failure
Environment=NODE_ENV=production
EnvironmentFile=/var/www/fileshare/apps/backend/.env
[Install]
WantedBy=multi-user.target
```
---
### 🐳 Hosting with Docker & Docker Compose (Any OS)
Run the entire stack (MySQL 8.0, Redis 7, Backend with FFmpeg, Frontend Next.js, and MinIO S3) with a single command:
```bash
# Build and launch all containers
npm run docker:up
# or
docker compose up --build -d
```
- **Frontend App**: `http://localhost:3000`
- **Backend API & Swagger**: `http://localhost:4000/api/docs`
- **MinIO S3 Console**: `http://localhost:9001` (User: `minioadmin` / `minioadminpassword`)
To view live container logs:
```bash
npm run docker:logs
```
To stop containers:
```bash
npm run docker:down
```
---
## 👥 Pre-Configured User Accounts
The database comes pre-seeded with the following active accounts:
| Role | Username | Email | Password | Initial Quota | Description |
|---|---|---|---|---|---|
| **Super Admin** | `Ecohub` | `ecohub@fileshare.local` | `Ecohub@520` | 100 GB | Full system control, quota manager, settings & audit logs |
| **Staff Officer**| `Ecohubstaff` | `ecohubstaff@fileshare.local` | `123` | 50 GB | Staff admin, user-wise file explorer, permission grants |
| **Normal User** | `ecouser` | `ecouser@fileshare.local` | `123` | 10 GB | Standard private vault, video streaming & sharing |
> *Tip: On the `/login` page, you can click on any of the 1-Click Instant Login cards to populate credentials immediately.*
---
## 🗄️ MySQL Direct Import File (`users_import.sql`)
For direct database seeding without running Node scripts, a standalone MySQL import script is provided in the root directory: [`users_import.sql`](file:///d:/Project/fileshareproject/users_import.sql).
### How to Import into MySQL:
#### Option 1: Via MySQL Command Line (Windows or Linux):
```bash
mysql -u root fileshare_db < users_import.sql
```
#### Option 2: Via phpMyAdmin or HeidiSQL:
1. Open **phpMyAdmin** or **HeidiSQL**.
2. Select the `fileshare_db` database.
3. Click the **Import** tab.
4. Choose the file [`users_import.sql`](file:///d:/Project/fileshareproject/users_import.sql) and click **Go**.
5. All three accounts (`Ecohub`, `Ecohubstaff`, `ecouser`), their Argon2id password hashes, and their root storage folders will be created immediately!
---
## 🧪 Automated Test Suite
Run the end-to-end integration and security test suites:
```bash
# Access control, IDOR, and permission tests
cd apps/backend
npm run test:e2e
```
**Tested Scenarios**:
1. Cross-tenant isolation (User B receives `404 Not Found` when requesting User A's private file).
2. Separation of `canPlay` and `canDownload` (User B can stream HTTP 206 chunks but receives `403 Forbidden` on download).
3. Immediate permission revocation (granting and revoking reflects instantly on protected routes).
4. Administrator-only routes protection (User A denied access to `/admin/*`).
5. Video processing pipeline (FFprobe probe, thumbnail generation, HLS master playlist & TS chunks).
---
## 📁 Repository Structure
```
fileshareproject/
├── docker-compose.yml # Multi-container orchestration (MySQL, Redis, Backend, Frontend)
├── start-windows.bat # 1-click Windows launcher
├── start-linux.sh # 1-click Linux launcher
├── .env.example # Comprehensive multi-environment configuration
├── packages/
│ └── shared/ # Shared TypeScript DTOs, enums, permission contracts
│ ├── package.json
│ └── src/index.ts
├── apps/
│ ├── backend/ # NestJS API Server
│ │ ├── Dockerfile # Linux Debian-slim container with FFmpeg & Prisma
│ │ ├── prisma/
│ │ │ ├── schema.prisma # MySQL Schema (Users, Folders, FileAssets, Permissions, Logs)
│ │ │ ├── migrations/ # Prisma migration history
│ │ │ └── seed.ts # Initial administrative bootstrap seeder
│ │ └── src/
│ │ ├── modules/
│ │ │ ├── auth/ # Argon2id password hashing, JWT access/refresh tokens
│ │ │ ├── storage/ # Cross-platform LocalStorage & S3Storage providers
│ │ │ ├── video/ # FFmpeg transcoder & HLS generator
│ │ │ ├── permissions/ # Granular permission engine (file, folder, user-level)
│ │ │ ├── files/ # Direct & chunked upload sessions, range streaming
│ │ │ ├── folders/ # Hierarchical logical folder management
│ │ │ ├── admin/ # User directory, storage metrics, audit trail
│ │ │ └── audit/ # Security event logger
│ └── frontend/ # Next.js 15 App Router Frontend
│ ├── Dockerfile # Multi-stage production container
│ └── src/
│ ├── app/
│ │ ├── dashboard/ # Storage gauges, quota bars, recent uploads
│ │ ├── files/ # File explorer, search, category filters, upload modal
│ │ ├── shared/ # Files shared with current user
│ │ ├── video/[id]/ # Adaptive HLS player with custom scrubber
│ │ └── admin/ # Users, user-files, permissions, storage, audit logs
│ └── components/
│ ├── video-player/ # HlsVideoPlayer component
│ ├── uploader/ # FileUploaderModal with drag-and-drop
│ └── file-manager/ # FileCard, FileRow, FilePreviewModal
└── storage/ # Default local storage volume (users/{id}/files/...)
```
---
## 🔒 Security Summary
- **No Public S3/Static URLs**: All file downloads, previews, and video streams pass through authentication and authorization guards.
- **Directory Traversal Immunity**: Storage paths use canonical keys and reject relative path components (`..`).
- **Input Sanitization**: Original file names are cleaned of special characters and illegal OS tokens.
- **Argon2id Hashing**: Industry-standard cryptographic password hashing.
- **Audit Compliance**: All administrative and file modifications are tracked in database audit records.