# Repository — Phase 3 outcome & verification report

**Date:** 2026-06-30
**Branch:** `fix/repo-test-issues-2026-06-29`
**Scope:** Phase 3 of `setup/repository_test_issues_mitigation.md` — the bulk
upload experience (SRS REPO-SP1 FRREP-DCI002/003, items 3.1–3.4). Builds on
Phase 0 (`repository_phase0_verification_2026_06_29.md`), Phase 1
(`repository_phase1_2026_06_29.md`) and Phase 2
(`repository_phase2_2026_06_29.md`).

All testing in the Docker stack (`docker-compose.yml`), one pytest invocation at
a time. New tests use `force_login`.

---

## A. Changes shipped

| # | Change | Files |
| --- | --- | --- |
| 3.1 | **Per-file real-time progress (FRREP-DCI003).** Additional bulk files no longer ride along synchronously in the form POST. Each selected file now stages out-of-band through the existing `StagedUploadView` via its own XHR — with an independent progress bar, Ready/Error state — exactly mirroring the primary file. The wizard collects the resulting staged tokens into a new hidden `bulk_files` JSON field; `DocumentSubmitView.form_valid` resolves each token with `_resolve_staged_upload` and ingests it as its own `Document`. A failed/aborted/missing file becomes a report row instead of sinking the whole run. The legacy `additional_files` multi-input is kept as the no-JS fallback (it still posts raw files, validated per-file at ingest). Staged blobs are discarded on success (`_discard_staged_upload` per consumed token). | `documents/forms.py` (`bulk_files`), `documents/views.py` (`form_valid` rewrite, `_parse_bulk_payload`, `_consumed_tokens`), `documents/templates/documents/document_submit.html` |
| 3.2 | **Batch summary report (FRREP-DCI002).** A bulk run now lands on a per-file outcome table (`bulk_submit_report.html`) instead of a single flash + redirect: each file is a **Queued** / **New version** / **Duplicate** / **Rejected** row with its `REPO-…` id (deep-linked) or its failure reason. Duplicates are reported (`Already in the repository as REPO-…`), never silently re-ingested; per-file `ValidationError`s (e.g. a disallowed format arriving via the no-JS path) are caught per file so the good files still land. A **Retry failed files** affordance lists exactly which files to re-upload and links back to the wizard (metadata is preserved on the form). Single-file submits keep the Phase 1 redirect-to-detail behaviour untouched. | `documents/views.py` (`_render_batch_report`), `documents/templates/documents/bulk_submit_report.html`, `documents/urls.py` (none — rendered in-place) |
| 3.3 | **Per-file metadata override (#8).** Each staged bulk file carries an optional `metadata` object inside `bulk_files`; `form.cleaned_data` is the shared template and `_effective_metadata` merges per-file overrides on top — only keys present **and** non-blank win, so a per-file edit can never blank a mandatory field. Title/year are pre-filled from `extract_file_metadata` (returned by the staged upload) as editable suggestions; the per-file editor also covers document-type, abstract, tags, and (server-side) authors + grant. `_guard_required` re-asserts the mandatory set + ≥1 author on each file's *effective* metadata before ingest, so Phase 1 enforcement isn't bypassed. | `documents/views.py` (`_shared_metadata`, `_effective_metadata`, `_guard_required`, `_ingest_one`), `document_submit.html` |
| 3.4 | **Replace/swap a staged file (#4).** The primary staged file gains a **Replace** button (re-opens the file picker; the existing `uploadFile` already discards the previous token and re-stages in place, preserving entered metadata). Each bulk file row gains **Replace** (discards the old staged token via `discardStaged`, then re-stages the new file into the same slot, keeping any per-file metadata typed) and **Remove** (abort + discard + drop). No wizard position is lost. | `document_submit.html` |

### Key decisions
- **Bulk reuses the staged-upload pipeline, not the chunked/resumable one.** Per-file progress mirrors the *primary* file, which uses `StagedUploadView` (one XHR with `upload.progress`), so bulk files use the same path for a consistent UX and the same `{token, name, metadata}` contract the view already consumes. The resumable pipeline (`ResumableStart/Chunk/Finalize`) remains available and also produces a staged token, so a future large-file bulk path can swap in without touching `form_valid`.
- **Single-file vs bulk are deliberately different terminal states.** A submission is "bulk" when there is more than one file *or* any unresolved bulk row. Single-file keeps the Phase 1 contract exactly (duplicate-resolution panel, redirect to detail) so no existing test changed; bulk always renders the report (HTTP 200, no redirect). This is why the duplicate **panel** stays single-file only — a bulk collision is a *report row*, which is the coherent batch behaviour (FRREP-DCI011 is not regressed: duplicates are still never silently ingested).
- **Overrides inherit, never blank.** `_effective_metadata` treats an absent or empty override key as "use the shared value". Mandatory fields therefore stay satisfied for every file (the shared set is already validated by `form.clean`), and `_guard_required` is a belt-and-suspenders re-check for the per-file author-override case.
- **No-JS fallback preserved.** With JavaScript off, Alpine never runs, the `additional_files` multi-input posts normally, and the legacy bulk path ingests them (now per-file resilient via the same report loop).

---

## B. Test results

New Phase 3 tests (`documents/tests/test_bulk_upload.py`, 10, all green):
- **3.1** — staged bulk tokens create one `Document` each; staged blobs discarded on success; a bad/missing token becomes a failure row without blocking the rest.
- **3.2** — a disallowed-format file (no-JS path) is a *Rejected* row while the good primary lands; a hash-colliding bulk file is a *Duplicate* row (existing id shown, not re-ingested); retrying the failed file alone with the `separate` disposition creates it.
- **3.3** — two bulk files with different per-file titles produce two docs with those titles while the un-overridden file + primary inherit the shared title; a title-only override still inherits the shared abstract/licence.
- **3.4** — `StagedUploadView` DELETE-then-POST swaps a mistaken staged file (old blob gone, new token + blob present); the wizard renders the Replace/bulk hooks (`replaceBulkFile`, `onBulkChange`, `name="bulk_files"`).

**Suite:** `docker compose exec -T web pytest apps/repository/` → **205 passed / 13 failed**
(was 195 passed / 13 failed after Phase 2 — +10 new passes, baseline failures
unchanged). The 13 are the pre-existing baseline, identical test names before and
after: 2 `test_ai_insights_live.py` (need a live `OPENAI_API_KEY`) + 11 django-axes
`client.login()` tests — `test_views.py` ×6 (qa-queue / staged-upload),
`test_followups.py` ×4 (incl. the older `test_dci002_bulk_submit_…`, which still
fails only at `client.login()`), `test_integration_endpoints.py` ×1. No new test
fails and nothing regressed.

> Operational note: run the repository suite as a **single** pytest invocation —
> a second concurrent run collides on `test_iilmp` and produces ~200 setup-time
> ERRORs (not code failures). If that happens, terminate stale connections and
> `DROP DATABASE test_iilmp` via `psql -U iilmp` in the postgres container, then
> re-run (see `feedback_test_db_concurrent_external.md`).

---

## C. Not done in Phase 3 (by design)

Deferred to Phase 4 per the mitigation plan: inline document reader (4.1),
author search facet / corpus reindex (4.2), advanced-search field wiring (4.3),
public-discoverability nav (4.4), and production media-serving verification
(4.5). Per-file **author** and **grant** overrides are accepted by the server
(`_effective_metadata`) but the bulk per-file editor UI surfaces the common
bibliographic fields (title, type, year, abstract, tags); authors/grant overrides
are a thin follow-up if the KM team wants them in the per-file panel. No login
wall was added to public read surfaces.