# Repository — Phase 0 outcome & verification report

**Date:** 2026-06-29
**Branch:** `fix/repo-test-issues-2026-06-29`
**Scope:** Phase 0 of `setup/repository_test_issues_mitigation.md` — small safety fixes
(0.1–0.5) + the verification sweep (0.6) that turns the 8 "NOT REPRODUCED" field
reports into either a confirmed prod issue or a green regression test.

All testing was run in the Docker stack (`docker-compose.yml`), one pytest
invocation at a time (test-DB contention). New tests use `force_login`.

---

## A. Code changes shipped (0.1–0.5)

| # | Change | Files | Test |
| --- | --- | --- | --- |
| 0.1 | **Restrict upload formats** — dropped `media` from the accepted categories (audio/video now rejected); kept `archive` for ZIP bundles. Added the **same category gate to the resumable start path** so chunked uploads can't bypass it. The direct-wizard path (`StagedUploadView.post`) already calls `_validate_repository_upload`, so it is covered too. | `apps/repository/documents/services.py` (`REPOSITORY_UPLOAD_CATEGORIES`, `_validate_repository_upload`, `start_resumable_upload`) | `documents/tests/test_upload_formats.py` |
| 0.2 | **Repository in the mobile menu** — added a collapsible Repository block to the topnav mobile panel, gated by `nav_show_repository` (same flag as desktop). Discover links public; Contribute behind auth; Manage behind repo-manager role. | `templates/components/topnav.html` | template parse-check (rendered surface; no unit assert) |
| 0.3 | **Removed `visibility` from the submit wizard** — dropped from `DocumentSubmitForm.Meta.fields` and stage-4 of `document_submit.html`; submit now hard-codes `Visibility.INHERIT`. Visibility stays editable on the edit/QA forms (`DocumentMetadataEditForm`). | `documents/forms.py`, `documents/views.py`, `documents/templates/documents/document_submit.html` | `documents/tests/test_submit_form_fields.py` |
| 0.4 | **Wired search audit logging** — `track_search_event(...)` is now called from `SearchView` (query + active filters + hit count) and `AdvancedSearchView`. Only fires on a real search, and **after** the top-queries empty-state so the current query doesn't pollute it. | `apps/repository/search/views.py` | `search/tests/test_audit.py` |
| 0.5 | **Fixed the empty grant dropdown (interim)** — `grant_reference` now renders as a `TextInput` instead of an empty `<select>`. `# TODO Phase 2.4` for the RIMS picker. | `documents/forms.py` | `documents/tests/test_submit_form_fields.py` |

### Decision logged for 0.1
Kept `archive` in the allowed list (ZIP/TAR bundles of related artefacts are a
legitimate repository upload). Only `media` (audio/video) was removed. If the KM
team confirms ZIP bundles are unwanted, drop `"archive"` from
`REPOSITORY_UPLOAD_CATEGORIES` — single-line change, the resumable gate reads the
same constant.

---

## B. Verification sweep (0.6) — the 8 "NOT REPRODUCED" items

| # | Claim | Verdict after verification | Evidence |
| --- | --- | --- | --- |
| **#5** | No cancel during upload | **Working — already covered** | `cancelUpload()` aborts the XHR + DELETEs the staged blob; `StagedUploadView.delete` and `abort_resumable_upload` clear the chunk dir. Regression already locked by `test_resumable_abort_view_clears_chunk_dir` and `test_staged_upload_delete_clears_token` (existing; both fail only on the unrelated django-axes baseline). |
| **#11** | Email & API ingestion not implemented | **Implemented — env not configured** | See *Config* below. API endpoint resolves at `repo_integration:rims_ingest` and enforces `repository_documents.ingest_via_rims`. Email poller keys off `EmailIngestMailbox` rows or `REPO_INBOUND_IMAP_*` env. The test env simply had neither set. |
| **#21** | Licence should be a dropdown | **Working** | `license_type` is a `Select` over the `License` (CC/OA) choices. New regression: `test_submit_form_license_is_a_choice_dropdown`. |
| **#24** | No success confirmation | **Working** | Single submit → `messages.success("Submission received: REPO-…")` + redirect to detail; bulk → count message + redirect to browse (`views.py` `DocumentSubmitView.form_valid`). Redirect path covered by existing wizard tests; the messages partial renders on both destinations. |
| **#25** | No unique human Document ID | **Working** | `public_document_id` = `REPO-{year}-{hex}`, rendered on the detail page (`document_detail.html:11,64`). New regression: `test_public_document_id_renders_on_detail`. |
| **#30** | View/download events not recorded | **Working** | `record_access_event` / `record_download_event` write `AccessEvent` / `DownloadEvent`. New regressions: `test_viewing_document_records_access_event`, `test_downloading_document_records_download_event`. |
| **#33** | Download delivers nothing | **Code correct — prod media issue** | `DocumentDownloadView` returns the real bytes (`200`, `application/pdf`, `Content-Disposition: attachment`, full 677 846 bytes verified against a live doc in the stack). New regression: `test_download_view_delivers_file_bytes`. The field report is a **deployment media-serving** problem — see *Media serving* below. |
| **#32** prereq | "Read online" blank | **Confirmed media-serving prereq; viewer is Phase 4.1** | "Read online" is a raw `<a href="{{ document.file.url }}">`. In the stack the media URL returns `200 application/pdf` (DEBUG serves `MEDIA_URL`). In prod (`DEBUG=False`) Django does **not** serve `/uploads/`, and WhiteNoise serves *static*, not *media* — so the link 404s/blanks. The inline viewer itself is Phase 4.1; the blank-page symptom is the media-serving gap below. |

### Media serving — root cause of #33 / #32 in production

Verified in the running stack (settings `config.settings.development`, `DEBUG=True`):

- `MEDIA_URL = /uploads/`, `MEDIA_ROOT = /app/uploads` (`config/settings/base.py:222`).
- `config/urls.py:35` only wires `static(MEDIA_URL, document_root=MEDIA_ROOT)` **when `DEBUG` is True**.
- A live document's `file.url` (`/uploads/repository/documents/...pdf`) returns **200 application/pdf** in the stack; the `DocumentDownloadView` returns the full file bytes as an attachment.

**Therefore the code is correct and the prod failure is infrastructure:**
1. On the EC2/prod host (`DEBUG=False`) nothing serves `/uploads/` — Django's
   dev static helper is off and WhiteNoise only serves `STATIC_ROOT`. The
   **`DocumentDownloadView` still works** (it streams bytes itself, not via the
   media URL) **provided the file exists on the prod disk**, but the raw
   `file.url` "Read online" link and any direct media URL will 404/blank.
2. **Fix (infra, not code):** add an nginx `location /uploads/ { alias …; }`
   block pointing at the prod `MEDIA_ROOT`, and confirm the migrated media is
   actually present on the box (the EC2 migration copied 2.7 GB of media — spot
   check the repository subtree).
3. **Follow-up (Phase 4.5):** switch `DocumentDownloadView` to
   `FileResponse`/streaming — it currently reads the whole file into memory.

---

## C. Config steps for #11 (email + API ingestion)

**API (RIMS → Repository), `POST repo_integration:rims_ingest` = `/repository/integration/ingest/`:**
- Caller must be authenticated **and** hold `repository_documents.ingest_via_rims`
  (or be a superuser). Provision a RIMS service account with that permission.
- Multipart body: a `file` field plus the RIMS payload; target collection slug
  defaults to `rims-closeout`, falling back to `reports`. **Seed one of those
  collections** or the endpoint returns 400.

**Email ingestion (poller `mail_ingest.poll_inbox`):**
- Either create an enabled `EmailIngestMailbox` row (per-collection inbox, set on
  the Collection form: "Allow submissions by email" + inbox address + allowed
  senders), **or** set the env vars `REPO_INBOUND_IMAP_HOST` / `_PORT` / `_USER`
  / `_PASSWORD` / `_FOLDER` to auto-register a "Default (env)" mailbox.
- Schedule/trigger `poll_inbox()` (Celery beat) to drain the inbox into
  `EmailIngestEvent` + Documents.

Both are implemented; the tester's "not implemented" was an unconfigured env.

---

## D. Test results

New Phase 0 tests (all green):
- `documents/tests/test_upload_formats.py` — direct + resumable format gate.
- `documents/tests/test_submit_form_fields.py` — visibility removed from submit / kept on edit; grant is a text input; licence is a choice dropdown.
- `documents/tests/test_phase0_verification.py` — Document ID on detail; access/download events; download delivers bytes.
- `search/tests/test_audit.py` — search + advanced-search write `SearchEvent`; bare landing writes nothing.

**Suite:** `docker compose exec -T web pytest apps/repository/` → **134 passed, 14 failed**
before fixing my own search regression; after the fix the search suite is
**15 passed**. The remaining **13 failures are pre-existing baseline**, confirmed
by `git stash`-ing all Phase 0 changes and re-running: the same 13 fail on a
clean tree (11 are the django-axes `client.login()` trap — see
`feedback_axes_client_login.md`; 2 are `test_ai_insights_live.py`, which need a
live `OPENAI_API_KEY`). The one test that passed on baseline and that my change
had briefly broken (`test_empty_search_seeds_top_queries_with_fallback`) is green
again after reordering the audit call to run *after* the top-queries empty-state.

---

## E. Not done in Phase 0 (by design)

Per the handoff, these stay for later phases and were **not** built:
registry-first author UI, RIMS grant picker, auto-classification, inline reader
(#32), bulk per-file progress/metadata, mandatory-field enforcement, collection
seeding/empty-state. No `LoginRequiredMixin` was added to search/detail (public
read access is intentional).