When reproducibility breaks: concrete failure modes and what practitioners can do

Four common, observed sources of reproducibility failure — disappearing proprietary models, opaque model behaviour, environment drift, and organisational churn — and practical, sociotechnical steps teams can take to reduce brittleness.

Why this matters

Reproducibility is not an abstract ideal; it is the operational bedrock of credible research and reliable software. In AI-assisted work the sociotechnical mix — people, processes, cloud services, and models — creates new, interlocking places where reproducibility can fail. Below I unpack four failure modes observed in practice and give concrete, practitioner-oriented guidance grounded in the reproducibility literature and discussions of the replication crisis.

1. Proprietary model disappearance

Description: When you rely on a hosted model or API, that external dependency can be deprecated, gated behind new terms, or removed entirely. The result: the exact inference pipeline you relied on no longer exists and cannot be rerun.

  • Practical guardrails: Where possible, prefer releasing or archiving the exact model weights, container images or a dockerised inference service alongside your work. If you must use a hosted API, capture its exact API version, the date and time of calls, request/response logs, and a representative sample of inputs and outputs.
  • Fallback plans: Keep a local alternative (open model, distilled version, or a simple rule-based baseline) that permits partial reproduction if the proprietary endpoint vanishes.
  • Sociotechnical step: Negotiate terms and document licensing and access constraints up front. Institutional memory about vendor commitments is often as important as technical archives.

2. Opaque model behaviour

Description: Closed models obscure internal updates or nondeterministic behaviour. Without transparency you cannot tell whether a difference in outputs is caused by a changed model, modified training data, or subtle runtime nondeterminism.

  • Practical guardrails: Record as much metadata as you can — model version, seed values, hardware type, and hyperparameters. Where possible, request model cards or change logs from providers. For in-house models, adopt internal release notes that record training data slices and augmentation steps.
  • Instrumentation: Capture deterministic checkpoints of the inference code and test-suite outputs; store a small, canonical testset with expected outputs so future divergence is detectable.
  • Sociotechnical step: Treat the model provider relationship like a component in your system architecture. Require change-notice guarantees, or formalise who in your organisation tracks provider updates.

3. Hidden environment drift

Description: Subtle changes in cloud images, library versions, or hardware affect results. Reproducing a run years later can fail because an image, a pip package release, or a GPU driver has changed.

  • Practical guardrails: Use immutable artefacts where feasible — container images, virtual machine snapshots, or full environment captures. Pin dependency versions and record the exact image digest or hash. For experiments, save the runtime environment (eg. a manifest produced by tooling such as pip freeze, conda list or similar) alongside experiment outputs.
  • Reproducible provenance: Link code, data, and environment in a single provenance record so future engineers can reconstruct the whole stack. Papers on failure reproduction emphasise preserving object state and external resource traces to make failures replayable.
  • Sociotechnical step: Build runbooks and simple checklists for environment capture into team processes so it becomes part of the workflow, not an afterthought.

4. Social and organisational churn

Description: People move on, incentives shift and undocumented handoffs erode institutional memory. Even if all technical artefacts exist, lost contextual knowledge — why a hyperparameter was chosen, which ad-hoc data filter was applied — can block faithful reproduction.

  • Practical guardrails: Combine short, focused documentation (what was done and why) with longer-form artefacts (experiment logs, commit history). Require that key decisions are recorded in pull requests or lightweight decision records.
  • Process: Schedule periodic “reproducibility audits” for critical projects: a colleague tries to rerun a release from artefacts and notes gaps. Use those audits to harden documentation and automation.
  • Sociotechnical step: Incentivise replication work. The biomedical literature and reproducibility reviews recommend funding and rewarding replication and audit efforts rather than only novel outputs.

Putting the pieces together — sociotechnical tactics

Fixing reproducibility requires combined technical and social practices. Technical tactics (archiving models and environments, pinning dependencies, deterministic seeds, canonical testsets) succeed only if paired with social practices (clear ownership, documented handoffs, contractual expectations with vendors, and incentives for replication). The replication crisis literature shows that incomplete disclosure of experimental details — from data preprocessing to software versions — is a recurring root cause; addressing it means changing what teams value and reward.

Quick checklist for teams

  • Archive: model artefacts + environment images + small canonical testset.
  • Document: versioned runbooks, decision logs, and vendor change notices.
  • Automate: CI checks that rerun key experiments on commit and flag drift.
  • Audit: periodic third-party or cross-team reproduction attempts and capture gaps.
  • Fallbacks: a named open-model or baseline and a plan if a provider changes or disappears.

Sources

Expanded from

This post expands on a point from Sociotechnical systems in software and AI: Reproducibility in the age of AI-assisted research.