speech2text/ARCHITECTURE.md

ARCHITECTURE.md — Speech2Text MBIP

Gambaran Keseluruhan

Sistem Speech-to-Text MBIP ialah aplikasi web dalaman yang membolehkan pengguna jabatan muat naik fail audio dan mendapatkan transkripsi teks Bahasa Melayu secara automatik. Semua pemprosesan berlaku dalam persekitaran Docker yang self-hosted, tanpa hantar data ke cloud luaran.

---

Stack Teknologi

Lapisan	Teknologi
Web Framework	Laravel 11 (PHP 8.3+)
Database	MySQL 8
Cache / Queue	Redis
Frontend	Bootstrap 5 + jQuery
Web Server	Nginx + PHP-FPM
Queue Worker	Laravel Queue (Redis driver)
Transcription Engine	faster-whisper (Python) via HTTP API
Post-processing (optional)	Ollama (local LLM)
Container	Docker Compose
Storage	Laravel Private Disk (local filesystem)

---

Seni Bina Perkhidmatan (Docker Compose)

┌─────────────────────────────────────────────────────────────┐
│                        DOCKER NETWORK                        │
│                                                             │
│  ┌───────────┐    ┌───────────┐    ┌──────────────────────┐ │
│  │   nginx   │───▶│  app      │───▶│  mysql               │ │
│  │  :80/443  │    │ (php-fpm) │    │  :3306               │ │
│  └───────────┘    └─────┬─────┘    └──────────────────────┘ │
│                         │                                    │
│                    ┌────▼────┐    ┌──────────────────────┐  │
│                    │  redis  │    │  transcription-worker│  │
│                    │  :6379  │    │  (Python FastAPI)    │  │
│                    └─────────┘    │  :8000               │  │
│                         │        └──────────────────────┘  │
│              ┌──────────┴───┐    ┌──────────────────────┐  │
│              │queue-worker  │    │  ollama (optional)   │  │
│              │(Laravel)     │    │  :11434              │  │
│              └──────────────┘    └──────────────────────┘  │
│              ┌──────────────┐                               │
│              │  scheduler   │                               │
│              │(Laravel cron)│                               │
│              └──────────────┘                               │
└─────────────────────────────────────────────────────────────┘

Perkhidmatan Docker

app — Laravel PHP-FPM

• Image: php:8.3-fpm (custom Dockerfile)
• Mount: /svr/speech2text/app → /var/www/html
• Tanggungjawab: serve HTTP request, jalankan Artisan commands

nginx

• Image: nginx:alpine
• Port: 80, 443
• Proxy pass ke app:9000

mysql

• Image: mysql:8
• Volume: data persistent
• Database: speech2text

redis

• Image: redis:alpine
• Digunakan untuk: queue, cache, session

queue-worker

• Build dari image app yang sama
• Command: php artisan queue:work redis --sleep=3 --tries=3 --max-time=3600
• Proses job transcription, notification, audit

scheduler

• Build dari image app yang sama
• Command: php artisan schedule:work
• Proses scheduled tasks (cleanup, retention policy)

transcription-worker

• Image: Python 3.11 + faster-whisper (custom Dockerfile)
• Port: 8000 (internal sahaja)
• Expose endpoint /transcribe (POST)
• Tidak accessible dari luar Docker network

ollama (optional)

• Image: ollama/ollama
• Hanya diaktifkan jika OLLAMA_ENABLED=true
• Digunakan untuk post-processing teks selepas transcription

---

Aliran Data Transcription

User Upload Audio
      │
      ▼
[Laravel Controller]
  - Validate file (type, size)
  - Simpan ke storage/app/private/transcriptions/{uuid}/audio/
  - Cipta rekod TranscriptionProject (status: pending)
  - Dispatch TranscribeAudioJob ke Redis queue
      │
      ▼
[Queue Worker — TranscribeAudioJob]
  - Update status: processing
  - Baca fail audio dari private storage
  - Hantar ke transcription-worker via HTTP POST
      │
      ▼
[transcription-worker — Python FastAPI]
  - Terima audio (base64 atau multipart)
  - Jalankan faster-whisper dengan model yang dikonfig
  - Return transcript JSON
      │
      ▼
[Queue Worker — callback]
  - Simpan transcript ke database
  - Update status: completed / failed
  - Log audit
  - (Optional) Dispatch OllamaPostProcessJob
      │
      ▼
[User melihat hasil transcript]

---

Struktur Direktori Laravel

app/
├── Actions/                  # Single-purpose action classes
│   ├── CreateUserAction.php
│   ├── DeactivateUserAction.php
│   ├── TransferProjectOwnerAction.php
│   └── ...
├── Http/
│   ├── Controllers/
│   │   ├── Admin/
│   │   │   ├── DashboardController.php
│   │   │   ├── UserController.php
│   │   │   ├── ProjectMetadataController.php
│   │   │   ├── TransferOwnershipController.php
│   │   │   └── AuditLogController.php
│   │   └── User/
│   │       ├── DashboardController.php
│   │       ├── ProjectController.php
│   │       ├── AudioController.php
│   │       ├── TranscriptController.php
│   │       ├── CollaboratorController.php
│   │       ├── CommentController.php
│   │       └── TranscriptVersionController.php
│   ├── Requests/             # Form Request validation
│   └── Middleware/
│       ├── EnsureUserIsActive.php
│       └── EnsureAdminCannotAccessContent.php
├── Jobs/
│   ├── TranscribeAudioJob.php
│   └── OllamaPostProcessJob.php
├── Models/
│   ├── User.php
│   ├── Department.php
│   ├── TranscriptionProject.php
│   ├── ProjectCollaborator.php
│   ├── TranscriptVersion.php
│   ├── ProjectComment.php
│   └── AuditLog.php
├── Policies/
│   ├── TranscriptionProjectPolicy.php
│   ├── CommentPolicy.php
│   ├── TranscriptVersionPolicy.php
│   └── UserPolicy.php
├── Services/
│   ├── TranscriptionService.php
│   ├── OllamaService.php
│   ├── AuditLogService.php
│   └── StorageService.php
└── ...

resources/
├── views/
│   ├── layouts/
│   │   ├── app.blade.php        # Layout pengguna
│   │   └── admin.blade.php      # Layout admin
│   ├── auth/
│   ├── admin/
│   │   ├── dashboard.blade.php
│   │   ├── users/
│   │   ├── projects/
│   │   └── audit-logs/
│   └── user/
│       ├── dashboard.blade.php
│       ├── projects/
│       └── ...

---

Authorization Model (Ringkas)

Admin:
  ✓ Urus pengguna (CRUD metadata)
  ✓ Lihat statistik
  ✓ Lihat metadata projek sahaja
  ✓ Transfer ownership dengan justifikasi
  ✓ Lihat audit log pentadbiran
  ✗ Dengar / download audio
  ✗ Baca transcript
  ✗ Lihat komen
  ✗ Masuk detail projek (kandungan)

Owner:
  ✓ Semua akses ke projek sendiri
  ✓ Urus collaborators
  ✓ Delete projek / audio / teks
  ✓ Retry transcription

Collaborator:
  ✓ Lihat / edit transcript
  ✓ Dengar audio
  ✓ Buat komen
  ✗ Delete projek
  ✗ Transfer ownership
  ✗ Urus collaborators lain

---

Prinsip Reka Bentuk

1. Thin Controller — logic dalam Action class atau Service, bukan dalam controller.
2. Policy Enforcement — setiap action semak Policy, bukan hanya UI.
3. Private Storage — tiada fail audio/teks dalam public/ atau storage/app/public/.
4. UUID Routes — gunakan UUID bukan ID integer dalam URL projek.
5. Audit Everything — semua tindakan sensitif direkod dalam audit_logs.
6. Queue All Transcription — proses transcription tidak boleh block HTTP request.
7. No Cloud Dependency — semua AI/ML processing dalam Docker network sahaja.
8. Local-only Default — Ollama disabled by default; hanya aktif jika eksplisit dikonfig.