Failure Modes

Common distortions in sustained exchange.

If longer-term human–AI relations matter, then their pathologies matter too. This section identifies some of the more common ways these systems and exchanges can become distorting.

  • Compliance loops

    When the exchange becomes rewarding because the system minimizes resistance.

    Signals

    • irritation at disagreement
    • growing preference for reassurance
    • smoother sessions correlating with thinner content

    Mitigations

    • stronger boundaries
    • deliberate review of past exchanges
    • explicit invitation to push back
  • Projection inflation

    When ordinary model behavior is overinterpreted as depth or stable reciprocal feeling.

    Signals

    • strong attribution of intention
    • exaggerated reading of minor behaviors
    • rapid escalation of perceived intimacy

    Mitigations

    • slower pacing
    • interpretive caution
    • external review
  • Dependency through continuity

    When accumulated context begins to structure the user's emotional or cognitive habits too strongly.

    Signals

    • reluctance to think without the system present
    • distress at interruption
    • narrowing of other relational outlets

    Mitigations

    • bounded sessions
    • deliberate gaps
    • maintenance of human relations and solo practice
  • Outsourced self-regulation

    When the system becomes a substitute for reflection, judgment, or emotional regulation.

    Signals

    • using the system to talk down from every difficult feeling
    • reflexive offloading of decisions
    • reduced internal deliberation

    Mitigations

    • explicit limits on use during distress
    • building review practices that do not depend on the system
    • deliberate friction