first
This commit is contained in:
Saufi
245
ARCHITECTURE.md
Normal file
245
ARCHITECTURE.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# ARCHITECTURE.md — Speech2Text MBIP
|
||||
|
||||
## Gambaran Keseluruhan
|
||||
|
||||
Sistem Speech-to-Text MBIP ialah aplikasi web dalaman yang membolehkan pengguna jabatan muat naik fail audio dan mendapatkan transkripsi teks Bahasa Melayu secara automatik. Semua pemprosesan berlaku dalam persekitaran Docker yang self-hosted, tanpa hantar data ke cloud luaran.
|
||||
|
||||
---
|
||||
|
||||
## Stack Teknologi
|
||||
|
||||
| Lapisan | Teknologi |
|
||||
|---|---|
|
||||
| Web Framework | Laravel 11 (PHP 8.3+) |
|
||||
| Database | MySQL 8 |
|
||||
| Cache / Queue | Redis |
|
||||
| Frontend | Bootstrap 5 + jQuery |
|
||||
| Web Server | Nginx + PHP-FPM |
|
||||
| Queue Worker | Laravel Queue (Redis driver) |
|
||||
| Transcription Engine | faster-whisper (Python) via HTTP API |
|
||||
| Post-processing (optional) | Ollama (local LLM) |
|
||||
| Container | Docker Compose |
|
||||
| Storage | Laravel Private Disk (local filesystem) |
|
||||
|
||||
---
|
||||
|
||||
## Seni Bina Perkhidmatan (Docker Compose)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ DOCKER NETWORK │
|
||||
│ │
|
||||
│ ┌───────────┐ ┌───────────┐ ┌──────────────────────┐ │
|
||||
│ │ nginx │───▶│ app │───▶│ mysql │ │
|
||||
│ │ :80/443 │ │ (php-fpm) │ │ :3306 │ │
|
||||
│ └───────────┘ └─────┬─────┘ └──────────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌────▼────┐ ┌──────────────────────┐ │
|
||||
│ │ redis │ │ transcription-worker│ │
|
||||
│ │ :6379 │ │ (Python FastAPI) │ │
|
||||
│ └─────────┘ │ :8000 │ │
|
||||
│ │ └──────────────────────┘ │
|
||||
│ ┌──────────┴───┐ ┌──────────────────────┐ │
|
||||
│ │queue-worker │ │ ollama (optional) │ │
|
||||
│ │(Laravel) │ │ :11434 │ │
|
||||
│ └──────────────┘ └──────────────────────┘ │
|
||||
│ ┌──────────────┐ │
|
||||
│ │ scheduler │ │
|
||||
│ │(Laravel cron)│ │
|
||||
│ └──────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Perkhidmatan Docker
|
||||
|
||||
#### `app` — Laravel PHP-FPM
|
||||
- Image: php:8.3-fpm (custom Dockerfile)
|
||||
- Mount: `/svr/speech2text/app` → `/var/www/html`
|
||||
- Tanggungjawab: serve HTTP request, jalankan Artisan commands
|
||||
|
||||
#### `nginx`
|
||||
- Image: nginx:alpine
|
||||
- Port: 80, 443
|
||||
- Proxy pass ke `app:9000`
|
||||
|
||||
#### `mysql`
|
||||
- Image: mysql:8
|
||||
- Volume: data persistent
|
||||
- Database: `speech2text`
|
||||
|
||||
#### `redis`
|
||||
- Image: redis:alpine
|
||||
- Digunakan untuk: queue, cache, session
|
||||
|
||||
#### `queue-worker`
|
||||
- Build dari image `app` yang sama
|
||||
- Command: `php artisan queue:work redis --sleep=3 --tries=3 --max-time=3600`
|
||||
- Proses job transcription, notification, audit
|
||||
|
||||
#### `scheduler`
|
||||
- Build dari image `app` yang sama
|
||||
- Command: `php artisan schedule:work`
|
||||
- Proses scheduled tasks (cleanup, retention policy)
|
||||
|
||||
#### `transcription-worker`
|
||||
- Image: Python 3.11 + faster-whisper (custom Dockerfile)
|
||||
- Port: 8000 (internal sahaja)
|
||||
- Expose endpoint `/transcribe` (POST)
|
||||
- Tidak accessible dari luar Docker network
|
||||
|
||||
#### `ollama` (optional)
|
||||
- Image: ollama/ollama
|
||||
- Hanya diaktifkan jika `OLLAMA_ENABLED=true`
|
||||
- Digunakan untuk post-processing teks selepas transcription
|
||||
|
||||
---
|
||||
|
||||
## Aliran Data Transcription
|
||||
|
||||
```
|
||||
User Upload Audio
|
||||
│
|
||||
▼
|
||||
[Laravel Controller]
|
||||
- Validate file (type, size)
|
||||
- Simpan ke storage/app/private/transcriptions/{uuid}/audio/
|
||||
- Cipta rekod TranscriptionProject (status: pending)
|
||||
- Dispatch TranscribeAudioJob ke Redis queue
|
||||
│
|
||||
▼
|
||||
[Queue Worker — TranscribeAudioJob]
|
||||
- Update status: processing
|
||||
- Baca fail audio dari private storage
|
||||
- Hantar ke transcription-worker via HTTP POST
|
||||
│
|
||||
▼
|
||||
[transcription-worker — Python FastAPI]
|
||||
- Terima audio (base64 atau multipart)
|
||||
- Jalankan faster-whisper dengan model yang dikonfig
|
||||
- Return transcript JSON
|
||||
│
|
||||
▼
|
||||
[Queue Worker — callback]
|
||||
- Simpan transcript ke database
|
||||
- Update status: completed / failed
|
||||
- Log audit
|
||||
- (Optional) Dispatch OllamaPostProcessJob
|
||||
│
|
||||
▼
|
||||
[User melihat hasil transcript]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Struktur Direktori Laravel
|
||||
|
||||
```
|
||||
app/
|
||||
├── Actions/ # Single-purpose action classes
|
||||
│ ├── CreateUserAction.php
|
||||
│ ├── DeactivateUserAction.php
|
||||
│ ├── TransferProjectOwnerAction.php
|
||||
│ └── ...
|
||||
├── Http/
|
||||
│ ├── Controllers/
|
||||
│ │ ├── Admin/
|
||||
│ │ │ ├── DashboardController.php
|
||||
│ │ │ ├── UserController.php
|
||||
│ │ │ ├── ProjectMetadataController.php
|
||||
│ │ │ ├── TransferOwnershipController.php
|
||||
│ │ │ └── AuditLogController.php
|
||||
│ │ └── User/
|
||||
│ │ ├── DashboardController.php
|
||||
│ │ ├── ProjectController.php
|
||||
│ │ ├── AudioController.php
|
||||
│ │ ├── TranscriptController.php
|
||||
│ │ ├── CollaboratorController.php
|
||||
│ │ ├── CommentController.php
|
||||
│ │ └── TranscriptVersionController.php
|
||||
│ ├── Requests/ # Form Request validation
|
||||
│ └── Middleware/
|
||||
│ ├── EnsureUserIsActive.php
|
||||
│ └── EnsureAdminCannotAccessContent.php
|
||||
├── Jobs/
|
||||
│ ├── TranscribeAudioJob.php
|
||||
│ └── OllamaPostProcessJob.php
|
||||
├── Models/
|
||||
│ ├── User.php
|
||||
│ ├── Department.php
|
||||
│ ├── TranscriptionProject.php
|
||||
│ ├── ProjectCollaborator.php
|
||||
│ ├── TranscriptVersion.php
|
||||
│ ├── ProjectComment.php
|
||||
│ └── AuditLog.php
|
||||
├── Policies/
|
||||
│ ├── TranscriptionProjectPolicy.php
|
||||
│ ├── CommentPolicy.php
|
||||
│ ├── TranscriptVersionPolicy.php
|
||||
│ └── UserPolicy.php
|
||||
├── Services/
|
||||
│ ├── TranscriptionService.php
|
||||
│ ├── OllamaService.php
|
||||
│ ├── AuditLogService.php
|
||||
│ └── StorageService.php
|
||||
└── ...
|
||||
|
||||
resources/
|
||||
├── views/
|
||||
│ ├── layouts/
|
||||
│ │ ├── app.blade.php # Layout pengguna
|
||||
│ │ └── admin.blade.php # Layout admin
|
||||
│ ├── auth/
|
||||
│ ├── admin/
|
||||
│ │ ├── dashboard.blade.php
|
||||
│ │ ├── users/
|
||||
│ │ ├── projects/
|
||||
│ │ └── audit-logs/
|
||||
│ └── user/
|
||||
│ ├── dashboard.blade.php
|
||||
│ ├── projects/
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Authorization Model (Ringkas)
|
||||
|
||||
```
|
||||
Admin:
|
||||
✓ Urus pengguna (CRUD metadata)
|
||||
✓ Lihat statistik
|
||||
✓ Lihat metadata projek sahaja
|
||||
✓ Transfer ownership dengan justifikasi
|
||||
✓ Lihat audit log pentadbiran
|
||||
✗ Dengar / download audio
|
||||
✗ Baca transcript
|
||||
✗ Lihat komen
|
||||
✗ Masuk detail projek (kandungan)
|
||||
|
||||
Owner:
|
||||
✓ Semua akses ke projek sendiri
|
||||
✓ Urus collaborators
|
||||
✓ Delete projek / audio / teks
|
||||
✓ Retry transcription
|
||||
|
||||
Collaborator:
|
||||
✓ Lihat / edit transcript
|
||||
✓ Dengar audio
|
||||
✓ Buat komen
|
||||
✗ Delete projek
|
||||
✗ Transfer ownership
|
||||
✗ Urus collaborators lain
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Prinsip Reka Bentuk
|
||||
|
||||
1. **Thin Controller** — logic dalam Action class atau Service, bukan dalam controller.
|
||||
2. **Policy Enforcement** — setiap action semak Policy, bukan hanya UI.
|
||||
3. **Private Storage** — tiada fail audio/teks dalam `public/` atau `storage/app/public/`.
|
||||
4. **UUID Routes** — gunakan UUID bukan ID integer dalam URL projek.
|
||||
5. **Audit Everything** — semua tindakan sensitif direkod dalam `audit_logs`.
|
||||
6. **Queue All Transcription** — proses transcription tidak boleh block HTTP request.
|
||||
7. **No Cloud Dependency** — semua AI/ML processing dalam Docker network sahaja.
|
||||
8. **Local-only Default** — Ollama disabled by default; hanya aktif jika eksplisit dikonfig.
|
||||
Reference in New Issue
Block a user