When AI assistance becomes dependency: a sociotechnical deep dive

Dependency is more than frequent use. Through a sociotechnical lens, problematic dependency appears when organisations lack monitoring, engineers cannot validate outputs, and systems become infrastructural without governance. Practical steps can prevent brittle systems and loss of institutional knowledge.

What do we mean by "problematic dependency"?

Thinking sociotechnically means seeing AI not only as a model or a tool, but as part of a system that includes people, processes and institutions. The worry is not simply that people use assistance a lot; it is that the organisation has let critical structures atrophy while the AI quietly becomes indispensable. That state — frequent use plus weak human oversight, eroded expertise and poor governance — is what I call problematic dependency.

How it looks in practice

Based on the concerns raised in the White Paper and sociotechnical theory, dependency shows up in predictable ways:

  • Outputs are accepted without review. Teams trust the AI because it usually helps, and stop doing routine checks that would catch drift or hallucinations.
  • Engineers cannot explain or validate what the AI produces. Code and prompts become the de facto source of truth while human expertise about domain assumptions fades.
  • The AI becomes infrastructural: embedded into pipelines, dashboards and downstream services without corresponding governance, version control or rollback plans.
  • Organisational monitoring is absent or inadequate: there are no metrics for AI quality in production, no incident response playbooks and no owner for long‑term behaviour.

Why this matters (briefly grounded in evidence)

Work in sociotechnical and social assistance literatures shows that dependency is not merely a behavioural label but a relational condition — it arises when systems and people shape each other over time and when care or assistance substitutes for capabilities that would otherwise be cultivated (see the PMC review on dependency). Similarly, policy analyses of "dependency syndrome" caution against accepting external supports without building capacity for autonomy. Translating those ideas to AI: an organisation that treats AI outputs as authoritative risks hollowing out human skills and losing institutional knowledge, while the technology becomes brittle under unexpected conditions.

Practical guidance to prevent and recover from dependency

The following steps are pragmatic, rooted in sociotechnical design principles and aimed at practitioners who operate or govern AI in organisations.

  • Map the sociotechnical dependencies. Identify where AI outputs feed other systems, decisions or human work. Treat each integration as a dependency to be managed: owners, SLAs, tests and fallbacks.
  • Keep humans in the loop for validation. Design review gates for critical outputs. Rotating reviewers helps keep multiple people familiar with system behaviour and prevents a single point of erosion in expertise.
  • Instrument and monitor. Define production metrics for quality, not just latency. Monitor drift, error patterns and downstream user corrections. Logging should capture prompts, model versions and salient context for later audit.
  • Maintain explainability and traceability. Ask for artefacts that allow engineers to explain why an output was plausible: prompt histories, retrieval traces or model rationale where available. Traceability supports debugging and learning.
  • Govern the infrastructure. Treat AI components like other infrastructure: versioning, CI/CD for model changes, staged rollouts, canaries and clear rollback procedures. Assign clear responsibility for decisions about model updates and decommissioning.
  • Preserve and rotate human expertise. Allocate time and incentives for engineers and domain experts to practise the tasks AI assists with. This is less about resisting tools and more about sustaining tacit knowledge so the organisation can evaluate AI outputs meaningfully.
  • Run dependency drills. Periodically simulate outages or degraded model behaviour. Confirm that teams can continue essential services without the AI, and document lessons on how to restore capabilities.
  • Embed governance into scale decisions. Scaling is tempting because AI can be integrated rapidly, but scale should follow governance readiness: ensure monitoring, review workflows and incident response are in place before widening deployment.

Checklist for teams

  • Do we have an owner for every AI‑dependent workflow?
  • Are key outputs routinely audited by humans who understand the domain?
  • Is there automated monitoring for quality and drift, with alerts to humans?
  • Are model versions, prompts and retrieval sources tracked for every production result?
  • Can the service operate in a degraded mode if the AI fails?
  • Do we intentionally train and rotate people to keep domain expertise fresh?

Final thoughts

Dependency is a sociotechnical phenomenon: it grows where technology substitutes for social practices without equivalent investment in people and governance. The solution is not to stop using assistance — that would be unrealistic and unhelpful — but to design for resilience. That means treating AI as an infrastructural component that requires ongoing care: monitoring, explanation, human expertise and governance. Doing so reduces brittleness and preserves the institutional knowledge that lets organisations adapt when the technology changes or fails.

Expanded from

This post expands on a point from Copilot or autopilot? When AI assistance becomes dependency — and whether that matters.

Sources