# IILMP — File Storage Utility

> Location: `apps/core/storage/`
>
> A single, versatile upload handler that manages all file types across every IILMP module.
> Organised storage paths, deep MIME validation (magic bytes, not just extension),
> per-module size limits, virus-safe sanitisation hooks, and clean DRF/form integration.

---

## Why a Custom Utility

Django's default `FileField` + `upload_to` string is too basic for a platform that handles:
- Thesis PDFs and research datasets (Repository)
- SCORM packages and H5P content (REP)
- Grant proposal bundles with multiple attachments (RIMS)
- Business plan documents and pitch decks (SME-Hub)
- Profile photos and alumni certificates (Alumni)
- Survey exports and M&E report templates (M&EL)

A single `UploadHandler` with a registry-driven approach means one place to add a new file type, one place to change size limits, and one place to plug in antivirus or cloud storage later.

---

## Directory Layout

```
apps/core/storage/
├── __init__.py                 ← exports: UploadHandler, FileValidator, upload_to_*
├── handler.py                  ← UploadHandler class — orchestrates the full pipeline
├── validators.py               ← FileValidator, MagicBytesValidator, FileSizeValidator
├── paths.py                    ← upload_to() callables per module
├── registry.py                 ← FILE_TYPE_REGISTRY: ext → FileTypeConfig
├── exceptions.py               ← UnsupportedFileType, FileTooLarge, MaliciousFile
└── tests/
    ├── __init__.py
    ├── test_handler.py
    ├── test_validators.py
    └── fixtures/               ← sample files for testing each category
        ├── sample.pdf
        ├── sample.docx
        ├── sample.jpg
        ├── sample.mp4
        └── sample.zip
```

---

## `registry.py` — File Type Registry

Every supported extension maps to a `FileTypeConfig` describing its category,
allowed MIME types, max size, and any post-processing hooks.

```python
# apps/core/storage/registry.py

from dataclasses import dataclass, field
from typing import Optional

@dataclass
class FileTypeConfig:
    category: str                       # 'image' | 'document' | 'data' | 'media' |
                                        # 'archive' | 'elearning' | 'code' | 'other'
    mime_types: list[str]               # accepted MIME types (trust-but-verify)
    max_size_mb: float                  # per-file size ceiling in MB
    icon: str                           # CSS icon class for UI display
    preview_supported: bool = False     # can the platform render a preview?
    thumbnail_required: bool = False    # generate thumbnail on upload?
    extract_text: bool = False          # run OCR / text extraction?
    notes: str = ""


FILE_TYPE_REGISTRY: dict[str, FileTypeConfig] = {

    # ── Images ──────────────────────────────────────────────────────────────
    ".jpg":  FileTypeConfig("image", ["image/jpeg"],               10,  "icon-jpg",  True,  True),
    ".jpeg": FileTypeConfig("image", ["image/jpeg"],               10,  "icon-jpg",  True,  True),
    ".png":  FileTypeConfig("image", ["image/png"],                10,  "icon-png",  True,  True),
    ".gif":  FileTypeConfig("image", ["image/gif"],                5,   "icon-gif",  True,  False),
    ".webp": FileTypeConfig("image", ["image/webp"],               10,  "icon-webp", True,  True),
    ".svg":  FileTypeConfig("image", ["image/svg+xml"],            2,   "icon-svg",  True,  False,
                            notes="SVGs are sanitised to remove embedded scripts before saving"),
    ".bmp":  FileTypeConfig("image", ["image/bmp"],                10,  "icon-img"),
    ".tiff": FileTypeConfig("image", ["image/tiff"],               20,  "icon-img"),
    ".ico":  FileTypeConfig("image", ["image/x-icon","image/vnd.microsoft.icon"], 1, "icon-img"),

    # ── Documents ───────────────────────────────────────────────────────────
    ".pdf":  FileTypeConfig("document", ["application/pdf"],       50,  "icon-pdf",  True,  True,  True),
    ".doc":  FileTypeConfig("document", ["application/msword"],    25,  "icon-doc",  False, False, True),
    ".docx": FileTypeConfig("document", [
                 "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
             ],                                                     25,  "icon-docx", False, False, True),
    ".xls":  FileTypeConfig("document", ["application/vnd.ms-excel"], 25, "icon-xls"),
    ".xlsx": FileTypeConfig("document", [
                 "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
             ],                                                     25,  "icon-xlsx"),
    ".ppt":  FileTypeConfig("document", ["application/vnd.ms-powerpoint"], 50, "icon-ppt"),
    ".pptx": FileTypeConfig("document", [
                 "application/vnd.openxmlformats-officedocument.presentationml.presentation"
             ],                                                     50,  "icon-pptx"),
    ".txt":  FileTypeConfig("document", ["text/plain"],            5,   "icon-txt",  True,  False, True),
    ".rtf":  FileTypeConfig("document", ["application/rtf","text/rtf"], 10, "icon-doc"),
    ".odt":  FileTypeConfig("document", ["application/vnd.oasis.opendocument.text"], 25, "icon-doc"),
    ".ods":  FileTypeConfig("document", ["application/vnd.oasis.opendocument.spreadsheet"], 25, "icon-xls"),
    ".odp":  FileTypeConfig("document", ["application/vnd.oasis.opendocument.presentation"], 50, "icon-ppt"),

    # ── Data / Structured ───────────────────────────────────────────────────
    ".csv":  FileTypeConfig("data", ["text/csv","text/plain","application/csv"], 50, "icon-csv"),
    ".json": FileTypeConfig("data", ["application/json","text/json","text/plain"], 10, "icon-json"),
    ".xml":  FileTypeConfig("data", ["application/xml","text/xml"], 10, "icon-xml"),
    ".yaml": FileTypeConfig("data", ["text/yaml","text/plain","application/x-yaml"], 5, "icon-yaml"),
    ".yml":  FileTypeConfig("data", ["text/yaml","text/plain","application/x-yaml"], 5, "icon-yaml"),

    # ── Archives ────────────────────────────────────────────────────────────
    ".zip":  FileTypeConfig("archive", ["application/zip","application/x-zip-compressed"], 200, "icon-zip"),
    ".tar":  FileTypeConfig("archive", ["application/x-tar"],      200, "icon-zip"),
    ".gz":   FileTypeConfig("archive", ["application/gzip","application/x-gzip"], 200, "icon-zip"),
    ".tar.gz": FileTypeConfig("archive", ["application/gzip"],     200, "icon-zip"),
    ".rar":  FileTypeConfig("archive", ["application/vnd.rar","application/x-rar-compressed"], 200, "icon-zip"),
    ".7z":   FileTypeConfig("archive", ["application/x-7z-compressed"], 200, "icon-zip"),

    # ── Audio ────────────────────────────────────────────────────────────────
    ".mp3":  FileTypeConfig("media", ["audio/mpeg"],               50,  "icon-audio"),
    ".wav":  FileTypeConfig("media", ["audio/wav","audio/x-wav"],  100, "icon-audio"),
    ".ogg":  FileTypeConfig("media", ["audio/ogg"],                50,  "icon-audio"),
    ".m4a":  FileTypeConfig("media", ["audio/mp4","audio/x-m4a"],  50,  "icon-audio"),
    ".flac": FileTypeConfig("media", ["audio/flac"],               200, "icon-audio"),

    # ── Video ────────────────────────────────────────────────────────────────
    ".mp4":  FileTypeConfig("media", ["video/mp4"],                500, "icon-video", True, True),
    ".avi":  FileTypeConfig("media", ["video/x-msvideo"],          500, "icon-video"),
    ".mov":  FileTypeConfig("media", ["video/quicktime"],          500, "icon-video"),
    ".mkv":  FileTypeConfig("media", ["video/x-matroska"],         500, "icon-video"),
    ".webm": FileTypeConfig("media", ["video/webm"],               200, "icon-video", True),

    # ── E-learning packages ──────────────────────────────────────────────────
    # SCORM and H5P packages are .zip files but treated differently
    ".scorm": FileTypeConfig("elearning", ["application/zip"],     200, "icon-scorm",
                             notes="Validated as SCORM by checking imsmanifest.xml inside zip"),
    ".h5p":  FileTypeConfig("elearning", ["application/zip"],      100, "icon-h5p",
                             notes="H5P package — checked for h5p.json inside zip"),

    # ── Markdown / Code ──────────────────────────────────────────────────────
    ".md":   FileTypeConfig("code", ["text/markdown","text/plain"], 2,  "icon-txt",  True),
    ".html": FileTypeConfig("code", ["text/html"],                  2,  "icon-code",
                             notes="Sanitised with bleach before storage"),
    ".py":   FileTypeConfig("code", ["text/plain","text/x-python"], 2,  "icon-code"),
    ".js":   FileTypeConfig("code", ["text/javascript","application/javascript"], 2, "icon-code"),
    ".css":  FileTypeConfig("code", ["text/css"],                   2,  "icon-code"),
}

# Global fallback size limit for unregistered extensions
DEFAULT_MAX_SIZE_MB = 10

# Module-level upload size overrides (can further restrict per module)
MODULE_SIZE_LIMITS: dict[str, float] = {
    "rims":       50,    # grant proposals, financial docs
    "rep":        500,   # SCORM packages, video lectures
    "repository": 200,   # theses, research datasets
    "mel":        25,    # reports, survey exports
    "alumni":     10,    # profile photos, certificates
    "smehub":     100,   # pitch decks, business plans, demo videos
}
```

---

## `exceptions.py`

```python
# apps/core/storage/exceptions.py

class StorageError(Exception):
    """Base class for all storage utility errors."""

class UnsupportedFileType(StorageError):
    """File extension or MIME type is not allowed."""

class FileTooLarge(StorageError):
    """File exceeds the allowed size limit."""

class MaliciousFile(StorageError):
    """Magic bytes do not match claimed MIME type — potential spoofing."""

class MissingPackageManifest(StorageError):
    """SCORM/H5P zip does not contain the required manifest file."""
```

---

## `validators.py`

```python
# apps/core/storage/validators.py

import magic
import zipfile
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
from .registry import FILE_TYPE_REGISTRY, DEFAULT_MAX_SIZE_MB
from .exceptions import UnsupportedFileType, FileTooLarge, MaliciousFile, MissingPackageManifest


class FileSizeValidator:
    """Raises ValidationError if the file exceeds max_mb megabytes."""

    def __init__(self, max_mb: float):
        self.max_mb = max_mb
        self.max_bytes = int(max_mb * 1024 * 1024)

    def __call__(self, file):
        if file.size > self.max_bytes:
            raise ValidationError(
                _("File size %(size).1f MB exceeds the %(max).0f MB limit.")
                % {"size": file.size / (1024 * 1024), "max": self.max_mb}
            )


class MagicBytesValidator:
    """
    Validates the true MIME type by reading the first 2KB of the file
    using python-magic. Never trusts Content-Type or the file extension alone.

    Raises MaliciousFile if the detected MIME type does not match any
    of the expected MIME types for this extension.
    """

    # Extensions where we check the zip manifest instead of raw MIME
    ELEARNING_EXTENSIONS = {".scorm", ".h5p"}
    SCORM_MANIFEST = "imsmanifest.xml"
    H5P_MANIFEST = "h5p.json"

    def __init__(self, extension: str, expected_mimes: list[str]):
        self.extension = extension.lower()
        self.expected_mimes = expected_mimes

    def __call__(self, file):
        file.seek(0)
        detected = magic.from_buffer(file.read(2048), mime=True)
        file.seek(0)

        # E-learning packages: must be a zip AND contain the right manifest
        if self.extension in self.ELEARNING_EXTENSIONS:
            if detected not in ("application/zip", "application/x-zip-compressed"):
                raise MaliciousFile(
                    f"Expected a zip archive for {self.extension} but detected {detected}"
                )
            self._validate_elearning_manifest(file)
            return

        # SVG: check for embedded script tags (XSS risk)
        if self.extension == ".svg":
            file.seek(0)
            content = file.read().decode("utf-8", errors="ignore")
            file.seek(0)
            if "<script" in content.lower():
                raise MaliciousFile("SVG contains embedded <script> — rejected for security.")
            return

        # General: detected MIME must be in expected list
        if detected not in self.expected_mimes:
            # Allow text/plain as a superset for text-based types
            if detected == "text/plain" and any(m.startswith("text/") for m in self.expected_mimes):
                return
            raise MaliciousFile(
                _(
                    "File content type '%(detected)s' does not match the expected "
                    "type for %(ext)s files. The file may have been renamed."
                )
                % {"detected": detected, "ext": self.extension}
            )

    def _validate_elearning_manifest(self, file):
        required = self.SCORM_MANIFEST if self.extension == ".scorm" else self.H5P_MANIFEST
        try:
            with zipfile.ZipFile(file, "r") as zf:
                names = [n.lower() for n in zf.namelist()]
                if not any(required in n for n in names):
                    raise MissingPackageManifest(
                        f"Package does not contain {required}. "
                        f"This does not appear to be a valid {self.extension} package."
                    )
        except zipfile.BadZipFile:
            raise MaliciousFile("File claims to be a zip but could not be opened as one.")
        finally:
            file.seek(0)


class FileValidator:
    """
    Composed validator: extension whitelist → size → magic bytes.
    Use as a Django form/model field validator or call directly in services.

    Usage:
        validator = FileValidator(module="rims")
        validator(uploaded_file)   # raises ValidationError on failure
    """

    def __init__(self, module: str, allowed_categories: list[str] | None = None):
        """
        module: one of 'rims', 'rep', 'repository', 'mel', 'alumni', 'smehub'
        allowed_categories: optional filter — only allow specific categories
                            e.g. ['image'] for profile photos
                                 ['document', 'data'] for grant attachments
                                 None = allow all registered types
        """
        self.module = module
        self.allowed_categories = allowed_categories

    def __call__(self, file):
        from .registry import MODULE_SIZE_LIMITS

        # 1. Determine extension
        import os
        _, ext = os.path.splitext(file.name.lower())

        # Handle .tar.gz double extension
        if file.name.lower().endswith(".tar.gz"):
            ext = ".tar.gz"

        # 2. Look up registry
        config = FILE_TYPE_REGISTRY.get(ext)
        if config is None:
            raise ValidationError(
                _("File type '%(ext)s' is not supported. Please upload a supported file format.")
                % {"ext": ext}
            )

        # 3. Category filter
        if self.allowed_categories and config.category not in self.allowed_categories:
            raise ValidationError(
                _("%(category)s files are not accepted here. Allowed: %(allowed)s.")
                % {
                    "category": config.category,
                    "allowed": ", ".join(self.allowed_categories),
                }
            )

        # 4. Size — use the tighter of module limit and per-type limit
        module_max = MODULE_SIZE_LIMITS.get(self.module, DEFAULT_MAX_SIZE_MB)
        effective_max = min(module_max, config.max_size_mb)
        FileSizeValidator(effective_max)(file)

        # 5. Magic bytes — true MIME type check
        try:
            MagicBytesValidator(ext, config.mime_types)(file)
        except MaliciousFile as e:
            raise ValidationError(str(e))
        except MissingPackageManifest as e:
            raise ValidationError(str(e))
```

---

## `paths.py` — Upload Path Generators

```python
# apps/core/storage/paths.py

import os
import uuid
from datetime import date
from django.utils.text import slugify


def _safe_filename(original: str) -> str:
    """
    Sanitise the original filename:
    - Strip directory traversal characters
    - Slugify the stem
    - Preserve the extension
    - Append a short UUID to guarantee uniqueness
    """
    _, ext = os.path.splitext(original)
    stem = slugify(os.path.splitext(os.path.basename(original))[0])[:60]
    uid = uuid.uuid4().hex[:8]
    return f"{stem}_{uid}{ext.lower()}"


def _dated_path(module: str, sub: str, filename: str) -> str:
    """
    Builds:  uploads/<module>/<sub>/<YYYY>/<MM>/<safe_filename>
    Example: uploads/rims/grants/2026/03/final-report_3a9f1c2b.pdf
    """
    today = date.today()
    safe = _safe_filename(filename)
    return os.path.join("uploads", module, sub, str(today.year), f"{today.month:02d}", safe)


# ── RIMS ────────────────────────────────────────────────────────────────────

def upload_to_grants(instance, filename):
    return _dated_path("rims", "grants", filename)

def upload_to_projects(instance, filename):
    return _dated_path("rims", "projects", filename)

def upload_to_scholarships(instance, filename):
    return _dated_path("rims", "scholarships", filename)

def upload_to_finance(instance, filename):
    return _dated_path("rims", "finance", filename)

def upload_to_operations(instance, filename):
    return _dated_path("rims", "operations", filename)


# ── REP ─────────────────────────────────────────────────────────────────────

def upload_to_course_content(instance, filename):
    return _dated_path("rep", "content", filename)

def upload_to_scorm(instance, filename):
    return _dated_path("rep", "scorm", filename)

def upload_to_h5p(instance, filename):
    return _dated_path("rep", "h5p", filename)

def upload_to_cv(instance, filename):
    return _dated_path("rep", "cv", filename)


# ── Repository ───────────────────────────────────────────────────────────────

def upload_to_repository(instance, filename):
    return _dated_path("repository", "documents", filename)

def upload_to_repository_thumbnails(instance, filename):
    return _dated_path("repository", "thumbnails", filename)


# ── M&EL ────────────────────────────────────────────────────────────────────

def upload_to_mel_reports(instance, filename):
    return _dated_path("mel", "reports", filename)

def upload_to_mel_surveys(instance, filename):
    return _dated_path("mel", "surveys", filename)


# ── Alumni ───────────────────────────────────────────────────────────────────

def upload_to_alumni_photos(instance, filename):
    return _dated_path("alumni", "photos", filename)

def upload_to_alumni_certificates(instance, filename):
    return _dated_path("alumni", "certificates", filename)


# ── SME-Hub ──────────────────────────────────────────────────────────────────

def upload_to_smehub_onboarding(instance, filename):
    return _dated_path("smehub", "onboarding", filename)

def upload_to_smehub_marketplace(instance, filename):
    return _dated_path("smehub", "marketplace", filename)

def upload_to_smehub_investment(instance, filename):
    return _dated_path("smehub", "investment", filename)
```

---

## `handler.py` — Main UploadHandler Class

```python
# apps/core/storage/handler.py

import os
from dataclasses import dataclass
from typing import Callable, Optional
from django.core.files.uploadedfile import UploadedFile

from .registry import FILE_TYPE_REGISTRY, FileTypeConfig
from .validators import FileValidator
from .exceptions import UnsupportedFileType


@dataclass
class UploadResult:
    """Returned by UploadHandler.handle() on success."""
    file: UploadedFile
    original_name: str
    extension: str
    category: str
    mime_types: list[str]
    size_bytes: int
    size_mb: float
    config: FileTypeConfig
    preview_supported: bool
    thumbnail_required: bool
    extract_text: bool


class UploadHandler:
    """
    Central upload orchestrator. Validates and prepares a file for saving.

    Usage in a service:
        from apps.core.storage.handler import UploadHandler
        from apps.core.storage.paths import upload_to_grants

        handler = UploadHandler(module="rims", allowed_categories=["document", "data"])
        result = handler.handle(request.FILES["attachment"])
        # result.file is the cleaned file ready to save to a model FileField

    Usage as a DRF or form validator:
        validator = UploadHandler.as_validator(module="rep")
        # pass as validators=[validator] on a serializer or form field
    """

    def __init__(
        self,
        module: str,
        allowed_categories: list[str] | None = None,
        extra_validators: list[Callable] | None = None,
    ):
        self.module = module
        self.allowed_categories = allowed_categories
        self.core_validator = FileValidator(module, allowed_categories)
        self.extra_validators = extra_validators or []

    def handle(self, file: UploadedFile) -> UploadResult:
        """
        Run the full validation pipeline and return an UploadResult.
        Raises django.core.exceptions.ValidationError on any failure.
        """
        # Core: extension whitelist + size + magic bytes
        self.core_validator(file)

        # Any extra validators (e.g. image dimension checks, virus scan hook)
        for v in self.extra_validators:
            v(file)

        # Build result metadata
        _, ext = os.path.splitext(file.name.lower())
        if file.name.lower().endswith(".tar.gz"):
            ext = ".tar.gz"

        config = FILE_TYPE_REGISTRY[ext]
        size_bytes = file.size

        return UploadResult(
            file=file,
            original_name=file.name,
            extension=ext,
            category=config.category,
            mime_types=config.mime_types,
            size_bytes=size_bytes,
            size_mb=round(size_bytes / (1024 * 1024), 2),
            config=config,
            preview_supported=config.preview_supported,
            thumbnail_required=config.thumbnail_required,
            extract_text=config.extract_text,
        )

    @classmethod
    def as_validator(cls, module: str, allowed_categories: list[str] | None = None):
        """
        Returns a plain callable suitable for use as a Django form/serializer validator.

        Example:
            class GrantAttachmentSerializer(serializers.Serializer):
                file = serializers.FileField(
                    validators=[UploadHandler.as_validator("rims", ["document", "data"])]
                )
        """
        instance = cls(module, allowed_categories)
        return instance.core_validator

    def get_type_info(self, filename: str) -> Optional[FileTypeConfig]:
        """
        Look up the FileTypeConfig for a given filename without validating.
        Useful for rendering the correct icon in a template.
        """
        _, ext = os.path.splitext(filename.lower())
        return FILE_TYPE_REGISTRY.get(ext)
```

---

## `__init__.py` — Public API

```python
# apps/core/storage/__init__.py

from .handler import UploadHandler, UploadResult
from .validators import FileValidator, FileSizeValidator, MagicBytesValidator
from .registry import FILE_TYPE_REGISTRY, FileTypeConfig
from .exceptions import (
    StorageError,
    UnsupportedFileType,
    FileTooLarge,
    MaliciousFile,
    MissingPackageManifest,
)
from .paths import (
    upload_to_grants,
    upload_to_projects,
    upload_to_scholarships,
    upload_to_finance,
    upload_to_operations,
    upload_to_course_content,
    upload_to_scorm,
    upload_to_h5p,
    upload_to_cv,
    upload_to_repository,
    upload_to_repository_thumbnails,
    upload_to_mel_reports,
    upload_to_mel_surveys,
    upload_to_alumni_photos,
    upload_to_alumni_certificates,
    upload_to_smehub_onboarding,
    upload_to_smehub_marketplace,
    upload_to_smehub_investment,
)

__all__ = [
    "UploadHandler",
    "UploadResult",
    "FileValidator",
    "FileSizeValidator",
    "MagicBytesValidator",
    "FILE_TYPE_REGISTRY",
    "FileTypeConfig",
    "StorageError",
    "UnsupportedFileType",
    "FileTooLarge",
    "MaliciousFile",
    "MissingPackageManifest",
    # path generators
    "upload_to_grants",
    "upload_to_projects",
    "upload_to_scholarships",
    "upload_to_finance",
    "upload_to_operations",
    "upload_to_course_content",
    "upload_to_scorm",
    "upload_to_h5p",
    "upload_to_cv",
    "upload_to_repository",
    "upload_to_repository_thumbnails",
    "upload_to_mel_reports",
    "upload_to_mel_surveys",
    "upload_to_alumni_photos",
    "upload_to_alumni_certificates",
    "upload_to_smehub_onboarding",
    "upload_to_smehub_marketplace",
    "upload_to_smehub_investment",
]
```

---

## Usage Examples

### In a model

```python
# rims/grants/models.py
from apps.core.storage import upload_to_grants, FileValidator

class GrantAttachment(models.Model):
    grant = models.ForeignKey("Grant", on_delete=models.CASCADE)
    file = models.FileField(
        upload_to=upload_to_grants,
        validators=[FileValidator(module="rims", allowed_categories=["document", "data"])],
    )
    uploaded_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.SET_NULL, null=True)
    uploaded_at = models.DateTimeField(auto_now_add=True)
```

### In a DRF serializer

```python
# rims/grants/serializers.py
from apps.core.storage import UploadHandler

class GrantAttachmentSerializer(serializers.Serializer):
    file = serializers.FileField(
        validators=[UploadHandler.as_validator("rims", ["document", "data", "archive"])]
    )

    def save(self, **kwargs):
        handler = UploadHandler(module="rims", allowed_categories=["document", "data", "archive"])
        result = handler.handle(self.validated_data["file"])
        # result.thumbnail_required → queue thumbnail task
        # result.extract_text → queue OCR task
        return result
```

### In a service

```python
# repository/documents/services.py
from apps.core.storage import UploadHandler
from apps.core.storage.paths import upload_to_repository

def ingest_document(user, file, metadata: dict) -> Document:
    handler = UploadHandler(module="repository")
    result = handler.handle(file)

    doc = Document.objects.create(
        title=metadata["title"],
        submitted_by=user,
        file=result.file,
        file_size_bytes=result.size_bytes,
        file_category=result.category,
        preview_supported=result.preview_supported,
    )

    if result.thumbnail_required:
        from .tasks import generate_thumbnail
        generate_thumbnail.delay(doc.id)

    if result.extract_text:
        from .tasks import extract_document_text
        extract_document_text.delay(doc.id)

    return doc
```

### In a template (file icon helper)

```python
# core/templatetags/storage_tags.py
from django import template
from apps.core.storage import FILE_TYPE_REGISTRY

register = template.Library()

@register.simple_tag
def file_icon(filename: str) -> str:
    import os
    _, ext = os.path.splitext(filename.lower())
    config = FILE_TYPE_REGISTRY.get(ext)
    return config.icon if config else "icon-file"

@register.simple_tag
def file_category(filename: str) -> str:
    import os
    _, ext = os.path.splitext(filename.lower())
    config = FILE_TYPE_REGISTRY.get(ext)
    return config.category if config else "other"
```

```html
<!-- In a template -->
{% load storage_tags %}
<span class="{% file_icon attachment.file.name %}"></span>
<span class="text-sm text-gray-500">{% file_category attachment.file.name %}</span>
```

---

## Settings Configuration

```python
# config/settings/base.py

# Upload directory (local dev / staging)
MEDIA_ROOT = BASE_DIR / "uploads"
MEDIA_URL = "/uploads/"

# Django upload limits
FILE_UPLOAD_MAX_MEMORY_SIZE = 10 * 1024 * 1024          # 10 MB buffer in RAM
DATA_UPLOAD_MAX_MEMORY_SIZE = 10 * 1024 * 1024          # 10 MB max request body
FILE_UPLOAD_PERMISSIONS = 0o644
FILE_UPLOAD_DIRECTORY_PERMISSIONS = 0o755

# python-magic (required by MagicBytesValidator)
# pip install python-magic
# On Ubuntu/Debian: apt-get install libmagic1
# On macOS: brew install libmagic
```

```python
# config/settings/production.py

# S3 / MinIO (django-storages)
DEFAULT_FILE_STORAGE = "storages.backends.s3boto3.S3Boto3Storage"
AWS_STORAGE_BUCKET_NAME = env("AWS_BUCKET_NAME")
AWS_S3_REGION_NAME = env("AWS_REGION")
AWS_DEFAULT_ACL = "private"                              # never public by default
AWS_S3_FILE_OVERWRITE = False
AWS_S3_OBJECT_PARAMETERS = {"CacheControl": "max-age=86400"}
MEDIA_URL = f"https://{env('AWS_BUCKET_NAME')}.s3.amazonaws.com/"
```

---

## Supported Extensions — Quick Reference

| Category | Extensions |
|---|---|
| Image | `.jpg` `.jpeg` `.png` `.gif` `.webp` `.svg` `.bmp` `.tiff` `.ico` |
| Document | `.pdf` `.doc` `.docx` `.xls` `.xlsx` `.ppt` `.pptx` `.txt` `.rtf` `.odt` `.ods` `.odp` |
| Data | `.csv` `.json` `.xml` `.yaml` `.yml` |
| Archive | `.zip` `.tar` `.gz` `.tar.gz` `.rar` `.7z` |
| Audio | `.mp3` `.wav` `.ogg` `.m4a` `.flac` |
| Video | `.mp4` `.avi` `.mov` `.mkv` `.webm` |
| E-learning | `.scorm` `.h5p` |
| Code / Text | `.md` `.html` `.py` `.js` `.css` |

Adding a new extension requires only one line in `FILE_TYPE_REGISTRY` — no changes to validators, paths, or the handler.

---

## Security & Operations

### Antivirus (ClamAV)

`apps/core/storage/validators.py` calls `scan_stream_optional()` for every file passing through `UploadHandler`, **regardless of source module** (Repository, REP, MEL, RIMS, Alumni, SME-Hub).

The scan only runs when **both** of these settings are present:

```python
# config/settings/production.py
CLAMD_ENABLED = True                  # bool, default True
CLAMD_TCP_HOST = env("CLAMD_TCP_HOST")  # e.g. "clamav.internal" — empty disables scanning
CLAMD_TCP_PORT = env.int("CLAMD_TCP_PORT", default=3310)
```

If `CLAMD_TCP_HOST` is empty, scanning is silently skipped — uploads still succeed but with no antivirus check. **Production environments MUST set both.** Stage / dev may leave them empty.

The wire protocol is ClamAV's `INSTREAM`, so any clamd-compatible daemon works (clamd, ClamAV-Cloud, etc.). No file is ever written to disk before scanning — failed scans raise `ValidationError` and abort the upload.

### Encryption at rest (FRREP-AC008)

Application-layer encryption of stored files is **out of scope**. Encryption-at-rest is delegated to the storage backend:

- **AWS S3 / MinIO:** SSE-S3 (AES-256, default for new buckets) or SSE-KMS for key-rotation policies. Set `AWS_S3_ENCRYPTION = "aws:kms"` in production settings.
- **On-prem disk:** LUKS or equivalent at the volume level.
- **Backups:** the same encryption scheme must apply to off-site backups (see `setup/architecture.md` operations runbook).

The platform stores file paths and metadata in Postgres; database-level encryption is the DBA's responsibility (TDE / cloud-provider managed encryption).

### TLS

Application-layer encryption-in-transit is enforced at the edge (nginx / load-balancer):

```nginx
listen 443 ssl http2;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
add_header Strict-Transport-Security "max-age=63072000" always;
```

Django settings `SECURE_SSL_REDIRECT = True` and `SESSION_COOKIE_SECURE = True` ensure no plaintext session traffic ever reaches the application.