The Feedback Loop Is the Thing
How Confident Should You Be? covered calibration - how much to trust outputs when the system is nominally healthy. This post covers recovery design: when outputs are wrong, how quickly does your organization notice, route the pain to the right owner, and convert the incident into learning that changes the system? Without that loop, "organizational memory" is just storage.
Without a loop, organizational memory is just storage - and storage goes stale.
Continuity under failure, not just rollback
At Hedera, aBFT was not only about surviving bad nodes. It was about continuing to produce correct outputs through disruption - not merely rolling back to a last good state, but maintaining forward progress under failure. The organizational analogue is not "we have a rollback script." It is whether wrong outputs reach someone who can fix the root cause before the damage compounds - and whether that fix updates the knowledge base, the runbook, or the model boundary so the class of failure gets rarer.
Volume without feedback is not memory
I learned the opposite lesson at SGInnovate building a talent database of more than ten thousand profiles. The database was technically full. It was not institutionally alive until feedback from recruiter outcomes flowed back to curation - who was good, who was miscategorized, which sources were noisy. Without that loop, volume became misleading precision. A knowledge graph without a healthy feedback loop is a filing cabinet with extra steps.
The database is rarely the bottleneck
Sau Sheong advocates for knowledge graphs and organisational memory. The architecture is directionally right. The constraint is rarely the database - it is whether feedback reaches the right people on a timeline that matters. An agent can populate a graph from tickets and incidents; if nobody acts on contradictions, you have automated stale institutional memory faster.
From pain to signal
Recovery design is not incident response theater. It is the minimum set of connections so that pain becomes signal. Rollback can stop bleeding; it does not teach you why the cut happened. The question is whether the next iteration of the system - human process, code, prompts, guardrails - incorporates what you paid for in the outage. If you do not have a tight loop from "wrong output" to "owner with authority to change the system," you will replay the same class of failure with different names.
Training without a loop
Training teams across cultures, I saw what happens when training has no feedback loop from learner outcomes to curriculum - the slides update slowly while the real world changes quickly. AI-augmented development has the same failure mode at scale: lots of generated artifacts, not enough signal that this pattern produced an outage, that prompt reliably produces unsafe edits.
Map the path before you need it in anger
At NUS, I have seen how failure becomes a crisis when the path from wrong output to responsible human is too long or tacit. When a standard operating picture lives only in someone's head, the system cannot route error - it can only surprise the next downstream consumer. Recovery design starts with mapping that path before you need it in anger: from wrong output to accountable owner to change in the system. If the path has more than one hop without a named owner at each hop, you are gambling.
Habits that make recovery real
Error routing
Before production, draw the chain: who sees the first symptom, who can stop the bleed, who can change the code or model boundary? If the answer is "the on-call engineer will figure it out," you have not designed recovery - you have outsourced panic.
Learning capture
Every AI-related failure should produce a short structured record: what happened, how fast we knew, what we changed - not to blame individuals, but to close the loop so the same class of failure gets rarer. If the record does not result in a change to process, tooling, or scope, the incident was entertainment.
Time-to-owner and repeat rate
Measure what matters for loops, not vanity metrics: time-to-owner (how long until someone with authority knew), time-to-change (how long until the system or scope actually shifted), and repeat rate (whether the same failure class shows up again under a new ticket ID). If repeat rate stays high, your feedback loop is performative - you are documenting without learning.
Feedback loop health checks
Quarterly, ask: are the loops we documented actually running? Are recruiters still feeding quality back into the database? Are postmortems changing prompts, tests, or ownership? Documentation without feedback is cosplay.
A blunt test
If you want a blunt test for whether recovery is real, pick one recent failure and trace it: how many hours from first wrong output to the person who could change the root cause - not the person who rebooted a service, but the person who could alter the prompt, the test, or the scope? If that number is measured in days, you do not have a feedback loop; you have a relay race where the baton drops.
Closing the series
Ownership, failure modes, calibration, recovery - together they are the readiness profile. The final post in this series steps back: what does it mean to shape work in that frame, not just ship it?
The Other Dimension · Part 5 of 6 · Previous: How Confident Should You Be? · Next: Shaping, Not Just Shipping