Preview fails with DIAGNOSTICS_INCOMPLETE on large Takeout after 0.1.3-pilot.1 deployment #26
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
A large Google Photos Takeout preview failed after deployment of 0.1.3-pilot.1 (Immich-Go 0.32.0-bridge.5). Track investigation and any resulting source, test, infrastructure and release changes here, separately from isolated malware scanning (#12).
Observed behavior
The preview is FAILED with DIAGNOSTICS_INCOMPLETE. Bridge reports that it could not retain the complete importer log. The retained log has 235,926 records: 170,209 INFO, 63,721 DEBUG, 1,991 WARN and five text/banner records; no ERROR/FATAL/PANIC records were found. The saved download is 68,390,321 bytes. It ends during ordinary duplicate warnings, with no final asset report and no affected-item manifest. Importer timestamps run from 2026-11-01T05:07:58.130280179Z through 2026-11-01T05:26:57.523330117Z (about 19 minutes). These timestamps cover the importer span, not ZIP verification or the entire queued attempt.
This is a different failure from the previous completed preview's singleton-stack error. A successful build/deployment and healthy HTTP endpoints did not establish that this full collection would complete. No actual import was started by the investigating agent. Private logs and media filenames are deliberately not attached here.
Previous baseline
Analysis: user-measured 28 minutes. Previous preview: app elapsed 40m25s; 33,410 assets in final report, 31,721 would process, 1,689 skipped; one stack error. A whole-server usage screenshot was retained locally, with approximate readings only; no memory measurement or confirmed graph timezone is available. Keep future analysis/preview/import timings and resource measurements separate, and record versions and options.
Deployed revisions
532f5202faInvestigation leads, not established causes
The diagnostics HTTP client allows five seconds per request and three attempts. Private capture checks filesystem free space per saved log record and writes capture status per chunk. Investigate slow network-backed storage, request acknowledgement/retry behavior, capacity, worker termination and connectivity using server-side evidence. The available importer log alone cannot distinguish them. Do not simply increase timeouts and declare the issue resolved.
Acceptance criteria
Both production attempts ended without a final report or complete diagnostics. The first retained 235,926 records over 18m59s; the retry retained 200,941 records over 14m17s. Neither retained log contains an ERROR/critical record. The retry confirms dry-run mode and the same 172-item destination inventory.
Live Docker events identify the retry preview worker exiting 1 at 2026-11-01T07:27:17Z, about two seconds after its last retained record, followed by removal. No OOM/kill event was supplied. This is the processor exit code, not proof of the importer child's exit status. The earlier worker exited 0; its operation was not identified in the event attributes.
User-supplied CIFS statistics for the staging sub2 share show zero failed reads, writes, flushes, query-info and set-info requests. There were two failed creates among 1,022,452 requests, cumulatively since October 23; without error codes/timestamps these cannot be assigned to either preview. Global reconnect counters cannot be assigned to Bridge; the other numbered share is sub1.
The same three local Takeout ZIPs completed an isolated preview in 31m27s: 33,410 total, 31,710 would process, 1,700 skipped, zero errors/pending, complete 239,140-record log and all affected items retained. This used AMD64, a simulated destination and local capture storage, so it does not reproduce production ARM64/SMB/destination conditions. The 11 extra skips versus the older server baseline are classified BETTER_VERSION_IN_LIBRARY, not established source-file loss.
Confirmed reporting gaps are fixed on published branch codex/issue-26-diagnostics-evidence at
31721527ff: retain worker exit and controlled Docker OOM evidence, terminal private-transfer route/HTTP status/timeout/connection category, and the first failed capture storage operation/error category. No raw error text, paths, credentials or response bodies enter support metrics. Incomplete diagnostics and failed-preview import gates remain enforced.Affected processor/diagnostics/API suites, focused race tests, vet, nine source contracts, documentation/publication exclusions, the changed-source secret scan and whitespace checks passed. The full archive reproduction preceded the additional reporting instrumentation; the pinned importer binary is unchanged. A real 15-second transfer-stall regression and a status-rename failure regression demonstrate possible failure mechanisms; neither establishes production causality.
Keep issue 26 open: the production root cause is unconfirmed. Await the retry Technical details JSON if available. The diagnostics source branch is now pushed. The user approved a 0.1.3-pilot.2 build and deployment to capture the underlying failure; preparation is in progress, with no new deployment yet. Private logs and media filenames are not attached.
Diagnostics pilot 0.1.3-pilot.2 is deployed on PROD-APP-02.
7bcffa5eb06566e3bdf872dd7c1fe29590b476b2oncodex/issue-26-diagnostics-evidence.3a05d05516b012c9dfd8fbbebe47adbd88c37528oncodex/bridge-0.1.3-pilot.2.0.32.0-bridge.5; valid stacking and failed-preview import gates are retained.Native backend/race/upload-client/importer checks and current source/importer/npm/native-image security policies passed. The first build (#268) stopped before publication on two newly introduced G104 findings. Making the failure-category helper explicitly record-only resolved them without modifying the security baseline; the retry passed all gates. The separate full dual-architecture source CI remains deferred/waiting for its runners; this is the established native ARM64 pilot path, not a full release declaration.
Independent anonymous registry reads verified all three digest/platform/version/source/user/locale checks. Public source archives/checksums matched the exact Bridge commit and importer build recipe. Only three encrypted image pins were assigned and verified through SOPS with the user's narrow authorization. All other encrypted configuration values and recipient metadata remained byte-for-byte identical. Exact staged-tree publication/workflow/pilot checks and Compose validation passed.
Installed immutable references:
git.sharedroot.net/jchapman/immich-bridge@sha256:1b576e044e6dd95937d6c403a69d7d9994e34d57813cae7cfe9b197de1b2002egit.sharedroot.net/jchapman/immich-bridge@sha256:675b480cdcd0a668839d3e84df52dd39cbafcce092457195096e24f39a7a36fbgit.sharedroot.net/jchapman/immich-bridge@sha256:fba78e3086d0a523e230976ea3b5556685d106194ebed5d3b40b287e388bf047The immediate pre-restart active-task guard passed, and the registered mount-dependent unit completed its restart/readiness wait. Subsequent public HTTPS
/health,/readyand/checks all returned 200. No browser control, direct server login or automatic preview/import start was used. Future deployments should select the exact infrastructure revision above; these changes are published on the task branches, with no main-branch merge performed.Keep #26 open. This release records worker exit/OOM evidence, private-transfer route/HTTP status/timeout/connection category, and the first storage operation/error category. It does not establish the original production cause. The next step is a user-started fresh preview and review of its Technical details plus private log if it fails; old attempts cannot acquire evidence that was not recorded.